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1. Reducing research waste the PenCLAHRC way 


Abbott Rebecca, Whear Rebecca, Thompson-Coon Jo, Bethel Alison, Rogers Morwenna, Moore Darren, Orr 
Noreen, Stein Ken 


PenCLAHRC, University of Exeter Medical School 


BACKGROUND. It has been estimated that ‘85% of all research is currently being wasted’ (Chalmers & Glasziou2009). The 
2014 Lancet series highlighted the issues around research waste in detail and provided recommendations for identifying 
research priorities, designing and conducting research, regulation of research and dissemination. For research 
prioritisation to be effective in reducing waste, the recommendations stated a need to be transparent and justifiable, to 
fully involve the ‘users’, and systematically address what is already known. 


AIMS. To describe and reflect on the National Institute for Health Research (NIHR) Collaboration for Leadership in 
Applied Health Research and Care (CLAHRC) South West (PenCLAHRC) research prioritisation process and its 
contribution to reducing research waste in the South West region. 


METHODS. Research prioritisation was a key feature of the pilot PenCLAHRC (2009-2014) (Whear 2015). In the current 
PenCLAHRC (2014-1018), we have adapted the process for identifying questions (clinical practice uncertainties) and how 
they are collected, the time involved in stakeholder engagement and how the stakeholders, including service users, are 
invited to engage in the various stages. Currently, 27 stakeholder organisations are part of the process. Engagement in 
identifying questions occurs through workshops designed to promote evidence-based practice with service users and 
clinicians around the region and through an open online web-tool. We also identify uncertainties through the CLAHRC’s 
themes and events to explore uncertainties on specific issues. Ranking of questions involves three phases over 12 weeks: 
two managed electronically and the final phase in a face to face meeting with all stakeholders. At this stage, questions 
have been developed by a team of systematic reviewers and information specialists into 4-6 page rapid evidence 
summaries, identifying what is already known on this topic and how important and locally relevant the issue is. Key 
aspects of the refined PenCLAHRC prioritisation process include: » Engagement with service users, clinicians and the 
public to identify uncertainties through evidence-based practice workshops ensures a research portfolio of clinically- 
relevant, locally tractable and patient-informed projects, * Involving representatives from all the partner organisations 
and service users in discussing and ranking uncertainties ensures that those that are prioritised meet the criteria agreed 
by the collaboration for meaningful and impactful research, + Incorporating knowledge of existing research evidence (in 
the form of systematically produced rapid evidence summaries) using the skills and experience of stakeholders ensures 
that prioritisation decisions build on existing research, « Identifying relevant research that is ongoing or planned, avoids 
research duplication and may foster collaboration, * Sharing uncertainties that are not prioritised for further 
development with other research organisations/networks e.g. NIHR and the South West NHS Research Design Service 
minimises waste in capturing clinical uncertainties, * Involving evidence consumers throughout the design and delivery of 
the prioritised research projects champions need and the value of research findings. 


RESULTS. The PenCLAHRC research prioritisation process has run seven times since 2009. Stakeholders have welcomed 
the iterative changes made to the process. Since 2009, more than 430 questions have been considered. Rapid evidence 
summaries of 73 of these have been prepared and discussed in face to face meetings. Of these, after a more focussed 
interrogation of existing evidence, 30 became ongoing research projects. The findings of at least three of these have 
already resulted in significant practice change in health service delivery within the region. 


LIMITS. Prioritisation continues to evolve as a result of feedback from our stakeholders. 


CONCLUSIONS. The PenCLAHRC Prioritisation model incorporates three of the four key recommendations put forward by 
Chalmers et al (2014) on‘ Increasing value, reducing waste’; engaging users and the public, identifying what is already 
known and establishing what research is ongoing . The model also reflects the values expressed in the NIHR Adding 
Value in Research framework by i) answering questions relevant to clinicians, patients and the public, ii) embedding 
partnership working across the NHS and academia at the inception of research projects and iii) in addition to publishing 
results in open access journals promotes the co-creation and communication of useable messages to those who are able 
to use them to change practice. 


Corresponding author: Rebecca Abbott — Universityof Exeter — Email: r.a.abbott@ exeter.ac.uk 
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2. Core concepts required for an educational training in EBP: a Delphi survey 


Albarqouni Loai*, Glasziou Paul', Hoffmann Tammy}?, Ilic Dragan’, Young Taryn’, Rydland Olsen Nina? 


Center for Research in Evidence Based Practice, Faculty of Health Science and Medicine, Bond University, Australia, *Monash University, 
Department of Epidemiology & Preventive Medicine, Melbourne, Australia, *Bergen University College, Norway 


BACKGROUND. Evidence-Based Practice (EBP) is an important part of quality healthcare.Thus, EBP is integrated in the 
curricula of undergraduate, postgraduate, and continuing education for health professionals of many disciplines. 
However, there are huge variations in the content of EBP training programs and curricula. For instance, while the Sicily 
statement suggests the coverage of the 5 EBP steps, substantial attention is given to the third EBP step “critical appraisal”, 
with less to the other steps. There is no consensus on the core concepts that should be taught as part of EBP training. 


AIMS. The aim of this study is to develop a core set of concepts that a clinician needs to learn as part of EBP. These 
concepts should be relevant to a clinician in any health discipline (e.g. medicine, nursing, allied health, pharmacy), and at 
any education Level (e.g. undergraduate, postgraduate, or continuing medical education). 


METHODS. The development of the core set of EBP concepts will consist of (i) generation of an intital list of EBP concepts 
based on a previously conducted systematic review and published EBP curriculum statements (e.g. Siciliy statement); (ii) a 
multi-round modified Delphi online survey (using Survey Monkey) to prioritise and gain consensus on the EBP core 
concept set; and (iii) a consensus meeting to finalise the core set of EBP concepts. Through email, list servs, and social 
media, along with snowballing, we will invite interested participants to register their interest in the Delphi survey. We will 
use a purposive sampling approach to identify teachers of EBP andclinicians with experience in teaching EBP. In each 
round of the Delphi survey, participants will be asked to rate the relative importance of each concept item listed as 
“omitted,” “mentioned,” “explained,” or “practised (with exercises)”. EBP concept items will be prioritised if >75 % of 
participants score an item’s importance within the range for one category. 


na 


RESULTS. The study is underway, with completion expected by October 2017, and preliminary results presented here.We 
generated an initial list of 214 EBP concepts, and through discussion by members of the steering group, reduced the List to 
86 concepts. We grouped these concepts into the relevant EBP steps. 180 individual registered their interest to participate 
in the Delphi survey: 99 were women (56%), 68 were medical (38%) and 40 were allied health professionals (22%), 134 
work in a university (74%) and 69 work in a hospital (38%), 84 had a clinical role (47%), and 138 currently responsible for 
teaching EBP (77%). We invited 180 participants for the first round of the Delphi survey. 


LIMITS. - 


CONCLUSIONS. This study will aim to achieve consensus from a large panel of experts from diverse health professional 
groups on the core set of EBP concepts that should be taught.This core set of EBP concepts will in turn improve the 
standardisation of EBP training and facilitate the evaluation of the the effectiveness of EBP trainings programs. 


Corresponding author: Albarqouni Loai — Bond University - Center Research in Evidence Based Practice 
(CREBP) — Email: loai.albarqouni@student.bond.edu.au 
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3. Why do authors derive new cardiovascular clinical prediction rules if they already exist? A 
mixed methods study 


Ban Jong-Wook?, Wallace Emmaz, Stevens Richard?, Perera Rafael? 


1£vidence-Based Health Care Programme, Centre for Evidence-Based Medicine, University of Oxford, Oxford, United Kingdom, 7HRB 
Centre for Primary Care Research, Royal College of Surgeons in Ireland, Dublin 2, Ireland, Nuffield Department of Primary Care Health 
Sciences, Medical Science Division, University of Oxford, Oxford, United Kingdom 


BACKGROUND. Researchers should examine existing evidence to determine the need for a new study. It is unknown 
whether developers evaluate existing evidence to justify new cardiovascular clinical prediction rules (CPRs). 


AIMS. We examined firstly whether authors cited existing cardiovascular CPRs in derivation studies. To understand why 
authors proceed to develop a new CPR when previous CPRs exist, we then studied the stated insufficiencies of existing 
cardiovascular CPRs, according to the authors of derivation studies. Lastly, a survey was conducted to examine why some 
authors cited existing cardiovascular CPRs and others did not. 


METHODS. Derivation studies of cardiovascular CPRs from the International Register of Clinical Prediction Rules for 
Primary Care were evaluated. We reviewed the introduction sections to determine whether existing CPRs were cited. 
Using thematic content analysis, the stated reasons for determining existing cardiovascular CPRs insufficient were 
explored. Study authors were surveyed via e-mail and post. We asked whether they were aware of any existing 
cardiovascular CPRs at the time of derivation, how they searched for existing CPRs, and whether they thought it was 
important to cite existing CPRs. 


RESULTS. Of 85 derivation studies included, 48 (56.5%) cited existing CPRs, 33 (38.8%) did not cite any CPR, and four (4.7%) 
declared there was none to cite. Content analysis identified five categories of existing CPRs insufficiency related to: (1) 
derivation (5 studies; 11% of 44), (2) construct (31 studies; 70%), (3) performance (10 studies; 23%), (4) transferability (13 
studies; 30%), and (5) evidence (8 studies; 18%). Authors of 54 derivation studies (71.1% of 76 authors contacted) 
responded to the survey. Twenty-five authors (46.3%) reported they were aware of existing CPR at the time of derivation. 
Twenty-nine authors (53.7%) declared they conducted a systematic search to identify existing CPRs. Most authors (90.7%) 
indicated citing existing CPRs was important. 


LIMITS. Our findings may not be generalizable to the CPRs in other clinical areas or more recently developed CPRs. 
Although published in 2014, the International Register of Clinical Prediction Rules for Primary Care only includes CPRs 
published up to 2009. It is important to consider that all derivation studies in our review predate the publication of 
TRIPOD statement and many of them predate the modern expectation that a systematic review of existing evidence 
should be done before any new research. In addition, we assessed only CPRs from the cardiovascular domain of the 
international register. The small number of the authors surveyed might have made it difficult to detect the associations 
between citation of existing CPR and the survey answers. 


CONCLUSIONS. Cardiovascular CPRs are often developed without citing existing CPRs although most authors agree it is 
important. Common justifications for new CPRs concerned construct, including choice of predictor variables or relevance 
of outcomes. Developers should clearly justify why new CPRs are needed with reference to existing CPRs to avoid 
unnecessary duplication. 


Corresponding author: Ban Jong-Wook — University of Oxford - Evidence-Based Health Care Programme — 
Email: jong-wook.ban@conted.ox.ac.uk 
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4. Point of care tools in preventing overuse of care 


Blaine Caroline 
BMJ 


BACKGROUND. Improvements in healthcare have led conversely to overtreatment and overdiagnosis. Action is required 
to identify patients at risk of overmedicalisation, to protect them from new medical invasion, and to suggest alternative, 
acceptable care. Point of care tools, such as BMJ Best Practice, are ideally placed to educate, support clinical decision- 
making and help practitioners to rapidly identify the medical issues related to overdiagnosis or overtreatment during the 
routine course of care. 


AIMS. We looked at the benefits in the current structure and evidence-base in BMJ Best Practice, evaluated this and 
identified areas of future development. 


METHODS. Selected issues associated with overuse of care were identified in BMJ Best Practice, a point-of-care web- and 
mobile-based tool for use by healthcare professionals. These were then reviewed as possible entry points into a virtual 
care pathway that would highlight relevant aspects and aid in averting overuse of care. Routine screening tests for 
people at average risk: prostate-specific antigen (PSA) for prostate cancer in the absence of shared decision-making; 
Diagnosis: imaging for nonspecific low back pain without red flags Treatment: antibiotics for suspected uncomplicated 
acute otitis media. 


RESULTS. The risks and benefits associated with the examined issues were all well-described in BMJ Best Practice in 
applicable subsections. New features such as the incorporation of Cochrane Clinical Answers and BMJ Rapid 
Recommendations allow for rapid assessment of the evidence base at the point of care promoting informed and shared 
decisions. The reviewed content lent itself to creation of a unique “Overuse of care pathway” by way of dedicated icons 
and/or extraction and aggregation of the relevant content into linked alerts. This would be on-the-go highlighting of the 
drawbacks of too much care. 


LIMITS. The new features proposed will need further user testing and validation before they could be made widely 
available. 


CONCLUSIONS. Web and mobile decision-support applications can raise awareness of overdiagnosis and overtreatment 
during the course of the clinical workflow. Examples of areas where this may be helpful include selected screening tests 
for people at average risk, and selected treatments for people with uncomplicated or chronic conditions. 


Corresponding author: Blaine Caroline — BMJ Group - Clinical Evidence — Email: cblaine@bmj.com 
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5. American College of Emergency Physicians clinical policies: the inter-rater reliability of quality 
of evidence assessment 


Brown Michael?, Kaji Amy2, Wall Steve?, Byyny Richard*, Kwok Heemun®, Haukoos Jason‘ 


1Michigan State University College of Human Medicine, *David Geffen School of Medicine at UCLA, *New York University School of 
Medicine, “University of Colorado School of Medicine, *University of Washington 


BACKGROUND. The Institute of Medicine’s standards for developing trustworthy clinical practice guidelines stipulate that 
guideline developers rate the level of confidence in the evidence underpinning recommendations. Investigators using the 
GRADE approach recently reported “good” inter-rater agreement (kappa = 0.68) for assessing the quality of evidence 
among guideline panel members after only a 1 hour training session. However, there remains uncertainty regarding the 
ability of panels to reliably assess the quality of a body of evidence, and even more uncertainty about how best to assess 
and synthesize the quality of evidence when developing clinical practice guidelines. 


AIMS. To assess agreement of article grading as part of the development of clinical policies for the American College of 
Emergency Physicians (ACEP) Clinical Policies Committee. 


METHODS. We performed a retrospective observational study to assess agreement among the team of methodologists for 
six recently developed clinical policies. The Clinical Policies Committee uses a team of five trained methodologists who 
are also actively engaged in the practice of emergency medicine. For each policy, critical questions are developed, 
followed by a formal literature search by a medical librarian. Using traditional systematic review methods, results of the 
initial search are culled by members of the guideline writing panel subcommittee to remove articles that are not relevant 
to the questions. The articles are then forwarded to the methodology team for formal Class of Evidence grading using a 
structured process and standardized grading rubric, resulting in one of four grade levels (I, Il, Ill, and X [e.g., major 
methodological limitations or not applicable to the critical question]) for each article. Two methodologists independently 
review and grade each article. Initial grades are then compared and a formal adjudication process is used to determine 
final grades for articles where grading discordance occurred. Final grades are then reported to the writing committee for 
synthesis and incorporation into the policy. Multi-rater and weighted kappas, and raw agreement were used to report 
agreement, and linear regression used to report trends. 


RESULTS. Between December 2015 and October 2016, a total of 515 articles were graded with a median of 90 articles 
(range: 7 to 140) per policy. The median multi-rater, weighted kappa across policies was 0.31 (range 0.14 to 0.43) and the 
median raw agreement was 0.70 (range: 0.20 to 1.00). Ten pairwise sets of independent grading by methodologists 
occurred with a median number of pairwise graded articles of 10 (range: 1 to 50); the median pairwise weighted kappa 
was 0.32 (range: -0.05 to 0.90); median raw agreement was 0.72 (range: 0.50 to 0.94). No trend was identified for kappa or 
raw agreement across policies (beta = -0.05, p = 0.07 for kappa; beta = -0.02, p = 0.28 for raw agreement) or when 
stratified by methodologists (beta range = -0.17 to 0.14, p range = 0.24 to 0.97 for kappa; beta range = -0.35 to 0.05, p 
range = 0.08 to 0.87 for raw agreement). 


LIMITS. This project was not developed a priori (i.e, conception and analyses were retrospective). The development of 
ACEP clinical policies is continually being refined and thus there were likely minor modifications to components of the 
grading process over the time period for which the six policies were developed. Precision around some agreement 
measures was limited by small numbers. 


CONCLUSIONS. Among a team of trained methodologists, agreement of quality of evidence assessment was modest, 
suggesting the importance of using two or more methodologists to grade each article, while also using of a formal 
adjudication process to optimize final class of evidence grading. 


Corresponding author: Brown Michael — University of Michigan Health System — Email: bbownm@msu.edu 
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6. Evidence-Based Research to put research in the context of existing knowledge: a systematic 
review 


Brunnhuber Klara’, Blaine Caroline’, Juhl Carsten Bogh?, Drachen Thea Marie’, Robinson Karen A.*, Bjerrum 
Merete, Christensen Robin’, Nykvist Hanna’, Jamtvedt Gro’, Lund Hans’ 


1Elsevier, 7The BMJ, *University of Southern Denmark,Johns Hopkins University, *University of Aarhus, ‘The Parker Institute, 7Western 
Norway University of Applied Sciences, ®Oslo and Akershus University College of Applied Sciences 


BACKGROUND. Studies evaluating research practice (meta-research studies) indicate that authors of scientific papers 
select their references based on strategic considerations and individual preferences, rather than a systematic approach, 
and that only a minor fraction of relevant earlier studies are ever cited. This may lead to waste in research as redundant 
and irrelevant studies are performed and published. The need for new research to be placed in the context of existing 
knowledge has been emphasized by eminent scientists for centuries. The design of new studies should be informed by a 
pre-planned systematic and transparent approach to using earlier research, and new results put in context of existing 
evidence; this is evidence-based research (1). A number of studies have evaluated the way authors of scientific papers 
refer to earlier research, however, there has not been a systematic approach to identify and characterize such research 
practice. (1) Lund H, Brunnhuber K, Juhl C, Robinson K, Leenaars M, Dorch BF, et al. Towards evidence based research. 
BMJ. 2016;355:15440. 


AIMS. We conducted a systematic review of scientific papers to address the following questions: 1. What is the current 
best available evidence on the impact of evidence-based research (EBR), i.e. are systematic reviews used to support 
decisions to plan, fund, approve, conduct, report and publish research? If so, are they effective? 2. What is the current best 
available evidence of the benefits of EBR? 3. What is the current best available evidence of harms caused by research 
that is not evidence-based? 4. What is the current best available evidence on the perceived or actual adverse effects of 
EBR? 


METHODS. A search was performed of MEDLINE, Embase, CINAHL, Web of Science, Social Sciences Citation Index, Arts & 
Humanities Citation Index, and Cochrane Methodology Register from inception to June 28th 2015 with an update 
performed prior to the EBHC Conference. Additionally, reference lists of all included studies were reviewed, and experts 
within the field of evidence-based research were also consulted to identify relevant studies. 


RESULTS. The electronic search yielded 25,997 citations, 421 additional studies were identified from experts’ reference 
lists. After reviewing abstracts, 288 studies met the criteria for full-text appraisal, yielding 90 included studies. The 
references of these 90 studies were checked for additional studies of interest. This identified a further 233 studies that are 
currently being screened in full text to identify which should be included in the study. So far, ten studies indicates that 
researchers continue to perform redundant research, five studies that authors only refers to a small fraction of similar 
earlier studies, three studies that authors neither used systematic reviews or used them poorly when arguing for a new 
study, and six studies indicates that authors either have a poor use or no use of systematic reviews when putting new 
results in context. In addition, six studies showed that supportive and significant studies were cited more often than non- 
supportive and non-significant studies. Further, three studies showed that authors selected references to earlier studies 
based upon preferences and strategic considerations. Thus, an answer to question one could be that the use of systematic 
reviews to support decisions to plan, fund, approve, conduct, report and publish research was rarely or never done. 


LIMITS. Identifying studies evaluating the behavior of researchers is difficult since no common search terms cover them 
all. This may be the reason why so many new and potentially relevant papers were identified from the reference lists. 
However, this first search related to evidence-based research has already helped to develop a much more focused search 
strategy. 


CONCLUSIONS. While conclusions cannot be drawn before the systematic review is complete, the work to date confirms 
the existing knowledge-base, i.e. researcher’s choice of references is based upon preferences and _ strategic 
considerations, and only a small fraction of relevant earlier studies is referred to when new studies are published. It 
seems that a pre-planned systematic and transparent approach to earlier research is rarely being used to support 
decisions to plan, fund, approve, conduct, report and publish research. 


Corresponding author: Lund Hans — Western Norway University of Applied Sciences - Center for Evidence- 
based Practice — Email: hans.lund@hvl.no 
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7. Teaching EBP: reflections from a transformative learning perspective 


Burls Amanda 
City University of London 


BACKGROUND. | have been teaching the skills for evidence-based health care since 1993 and have had several other 
roles in this area including being the Director of the Critical Appraisal Skills Programme (CASP) and Director of 
Postgraduate Programmes in Evidence-Based Health Care at the University of Oxford. Although students evaluate my 
teaching well and | have often been asked to train others to teach or explain how | teach, my approach based on 
personal experience and reflection with implicit assumptions rather than being informed by explicit research evidence 
about effective teaching or learning theories. Recently | had the opportunity to address this deficit in theoretical learning 
background by taking a Postgraduate Diploma in Learning and Teaching in Higher Education which included producing a 
portfolio of reflective practice. My portfolio was entitle "Reflections on Teaching Evidence-Based Practice" and used the 
lens of transformative learning theory. 


AIMS. The aim of my presentation will be to share with other teachers and developers of Evidence-Based Health Care, the 
lessons | have learned about the effective teaching of evidence-based practice, informed by Mezirow's Transformational 
Learning Theory (also called Transformative Learning Theory). 


METHODS. As part of the process of learning and producing the portfolio, | undertook in-depth qualitative interviews with 
a purposive sample of former students on the Masters in Evidence-Based Health Care from the University of Oxford, 
about their learning experiences using a transformative learning perspective. 


RESULTS. In my talk | will briefly talk about the key concepts and evidence about transformative learning. | will argue 
that evidence-based practice is fundamentally transformative in purpose as it requires the questioning of our 
assumptions and expectations about the world. | will propose that we should distinguish between transformative and 
transformational learning (the terms are currently used interchangeably in most of the literature). | will report the 
findings of the interviews with eight former students and the areas in which they match or diverge from transformative 
learning theory. Key messages that former students consistently reported as transformative included collaborative 
learning, a (personal) problem-based approach, interactivity and the modelling of a questioning evidence-based 
approach by teachers. 


LIMITS. The work reported was a personal reflective, theoretically informed, exercise in which | sought to articulate my 
implicitly learnt lessons for effective teaching of evidence-based health care rather than a scientific piece of research 
attempting to prove or refute a hypothesis. Consequently what | report will be full of my biases and beliefs and the 
lessons learnt which are relevant to my teaching approach may not be generalizable to others. 


CONCLUSIONS. Transformative learning theory captures much of what |, and many other teachers of evidence-based 
practice, hope our students will learn. Conceptualizing learning in this way is liberating for the teacher and revolutionary 
for the learner. 


Corresponding author: Burls Amanda — City University London — Email: amanda.burls.1@city.ac.uk 
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8. Are we doing enough to disseminate if we don't engage in the hard questions? 


Butler Mary 


University of Minnesota 


BACKGROUND. Multiple sclerosis (MS) is a variably debilitating and currently incurable disease. Available MS 
medications to slow disease progression are expensive and not without potential adverse effects. How long disease 
modifying treatments (DMT) should be administered remains an open and controversial question. Some patients cannot 
tolerate any available DMTs, but if a tolerable drug regime is determined, treatment generally continues until the 
individual reaches a disease stage where DMTs no longer help, i.e., when a person is nonresponsive to the medication due 
to disease progression. However, determining when DMT is no longer helpful is challenging. Thus, to inform potential 
shared decision-making, major questions of interest are whether or not DMTs for MS alter the natural history of the 
disease in the long run and when (if ever) to discontinue DMTs. 


AIMS. The aim of the underlying review was to assess long-term benefits and harms of discontinuing MS medications, and 
both individual and provider values, beliefs, and preferences for doing so. 


METHODS. The research questions arose from a topic development and refinement process that included discussions 
with a wide range of stakeholder key informants, including people with MS. A hybrid Agency for Healthcare Research and 
Quality Evidence-based Practice Center Program systematic review was undertaken to assess 1) the epidemiological 
evidence for long-term benefits and harms of DMT, and 2) the empirical evidence, quantitative and qualitative, that 
would inform understanding individual and provider values, beliefs, and preferences, and the implications for shared 
decision-making. A conceptual model, split into 3 broad conceptual areas of intra- and inter-personal psychological 
factors or process and shared decision-making, was used as a framework to organize the empirical evidence. The draft 
review underwent a peer review and 1-month posting for public comments before the final report was published. A table 
of responses to all peer review and public comments was published as supplementary material. 


RESULTS. Epidemiological evidence to inform decisions for optimal discontinuation of DMTs was mostly non-existent; 
what was available was sparse and low quality. In the absence of evidence, providers and patients are left with little to 
inform their preferences and guide their decisions regarding when to discontinue treatments. While little literature was 
available to populate the conceptual map for preferences and shared decision-making, each of the three major 
conceptual areas (inter-, intra-personal, and shared decision-making) was at least partially represented. No study directly 
asked why people are reluctant to discontinue when treatment seems no longer effective. However, as a whole the 
literature set provides some insight into the conflicting forces for stopping DMTs. Despite early peer involvement in the 
topic development and refinement stage and transparent Agency for Healthcare Research and Quality methods, an 
erroneous belief within the MS clinic community that the report was advocating no reimbursement for DMTs spurred an 
organized campaign to silence the report results. The written campaign ranged from reasoned concern to vitriolic. Two 
years later, a randomized controlled trial is currently recruiting a carefully selected class of patients to test the efficacy 
and safety of discontinuing DMTs. This study is being conducted by one of the report’s harshest public comment critics. 


LIMITS. This experience represents an N of 1 study. 


CONCLUSIONS. The ecology of evidence must accommodate the full range of evidentiary questions, including not only 
what works or not and for whom, but also for how long? We have been good at asking questions about starting 
treatments, but when to stop has been less examined. Stopping treatment because a disease course has progressed, and 
when there isn’t a next alternate, is difficult. As a community, are we doing enough to disseminate if we do not also 
engage in the skill development and creation of venues to initiate and foster difficult dialogue between patients, 
providers, researchers, policy-makers, all stakeholders about deeply held values and realistic limitations of healthcare, 
particularly where this begs acknowledgement of our human mortality requires direct engagement with uncertainty and 
fear? This session starts the conversation with a few suggestions. These draw in part from cognitive psychology and 
communication studies theory (e.g. specific strategies such as “debunking misinformation” through affirming shared 
values), and related conversations being held in other science fields. Hopefully, these suggestions will seed further 
collaboration on solutions to derive and translate more impactful evidence. 


Corresponding author: Butler Mary — University of Minnesota — Email: butL0092@umn.edu 
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9. Evaluating the impact of systematic reviews in disability and rehabilitation research using 
bibliometric mesures 


Cai Xinsheng, Mangrum Rikki, Garfinkel Steven 


American Institutes for Research 


BACKGROUND. Spinal cord injury (SCI), traumatic brain injury (TBI), and burn injury (BURN) are three costly injuries that 
affect millions of individuals around the globe. Systematic reviews on effective treatments and rehabilitation are 
important to improve research, clinical care and policy leading to improved quality of lives for individuals with 
disabilities. The Model Systems in SCI, TBI and Burn are clinical centers of excellence funded by the National Institute on 
Disability, Independent Living and Rehabilitation Research (NIDILRR), Administration for Community Living, US 
Department of Health and Human Services. Supported by the Model Systems Knowledge Translation Center (MSKTC), 
they conduct systematic reviews of evidence in SCI, TBI and Burn. 


AIMS. Demonstrate the value of bibliometric measures for evaluating the impact of systematic reviews and the quality of 
studies included in systematic reviews. 


METHODS. Researchers conducted a bibliometric analysis of systematic reviews using Web of Science databases. Impact 
was measured by examining levels and trends in citation and journal quality indicators. In addition, the quality and 
impact of the original studies included the systematic reviews were examined through citation levels and metrics such as 
journal impact factor scores, journal ranking, and author h-index trends. 


RESULTS. The MSKTC-supported systematic reviews in SCI, TBI and Burn were cited consistently, in some cases with 
increased frequency in recent years. For example, one review accrued limited citations for a few years, but has been cited 
17 times in each of the last two reporting years. Reviews were cited most frequently in Rehabilitation journals and 
penetrated into related topical fields such as sport sciences, clinical care specialties, gerontology, or health policy. They 
crossed research and practical audiences, and included non-English language publications. This suggests that these 
reviews have had broad impact outside core topical fields and may impact research, clinical care, and policy. The analysis 
also revealed that the articles included in the reviews tend to be published in highly ranked, higher impact journals and 
are frequently cited with averages ranging from 23 to 54 citations. These articles also tend not to exhibit much self- 
citation. 


LIMITS. The level of citation is tied to the number of years since publication. Newer publications accrue limited citations 
due to publication lag and a lag in reporting in the Web of Science; the typical lag time was 2 years. 


CONCLUSIONS. Bibliometric analysis is a valuable tool for evaluating the quality and impact of systematic reviews. 
Because of publication and citation reporting lag time, researchers may need to wait at least two years before beginning 


a bibliometric evaluation. 
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10. Completeness of reporting in observational studies: effects of prior formal training in research 
methodology 


Calvache José Andrés?, Arroyave Yeni’, Bravo-Pena Mary, Barona-Fong Luis? 


1Department of Anesthesiology, Universidad del Cauca, Popayan, Colombia and Department of Anesthesiology, Erasmus University 
Medical Centre Rotterdam, Rotterdam, The Netherlands 


BACKGROUND. Observational studies account for the majority of publications in health care. The STROBE statement 
(Strengthening the Reporting of Observational Studies in Epidemiology) was published in 2007 with the aim of improving 
the reporting and completeness of observational research. In developing countries like Colombia, there is still an limited 
number of researchers with formal training in methodology or research-related areas. 


AIMS. To determine the completeness of reporting of observational studies published in Colombian Journal of 
Anesthesiology between 2000 and 2013, using the STROBE statement. In addition, to test if the presence of an author with 
formal training increases the average completeness of the observational studies. 


METHODS. Cross-sectional study. All observational studies published in RCA between 2000 and 2013 were included. We 
included cohort studies, case-control studies and cross-sectional (or prevalence) studies. Studies without a clear, well- 
defined methodological design, editorials, letters to the editor, reflection articles, narrative or systematic reviews of the 
literature, case reports and case series were excluded. A random selection of 40% of the total (23 studies) was selected 
and annual representation in the sample was ensured by means of proportional fixation by year. Using STROBE 
Statement each included study was analyzed by two researches independently. Each item of the statement was qualified 
as “met”, “unmet” or did “not apply”, in accordance with the specific research design. In cases of difficulty, the rating was 
discussed individually with a referee. The primary outcome was the completeness of the report of each study. 
Additionally, other variables were extracted, such as sample size, number of authors, number of professors, year of 
publication and presence or not of authors with Masters (MSc) or Doctoral degrees (PhD). Descriptive summaries, time 
series analyses and multiple linear regression were used. 


RESULTS. Twenty-three observational studies were analyzed. The mean completeness was 57%, 95% Cl [48-66%]. The 
sections for which greater completeness was documented were “Title” and “Introduction” (89% each). Sections with 
lowest completeness were “Methods” and “Results” (50%). During the 13-year study period there was a very slow increase 
in completeness. The adjusted completeness of the reports that included an author with a Master's or PhD degree was 
higher (B=10.3, SE=3.1 p=0.03). 


LIMITS. The main limitation was failure to study the entire population of observational studies, which reduce the 
confidence of our estimates. Potential type II error could be present. However, our objective was to document its 
existence in order to promote improvement in completeness of reporting. 


CONCLUSIONS. Completeness of observational studies published in Colombian Journal of Anesthesiology is 57%. Title 
and Introduction are the sections with greatest completeness, while Methodology and Results are the ones with the 
greatest deficiencies. Formal degree-training in research influence positively the completeness of reporting. 


Corresponding author: Calvache Espana Jose Andres — Universidad del Cauca - Email: jacalvache@ unicauca.edu.co 
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11. Comparison of imprecision assessed by GRADE and trial sequential analysis in systematic 
reviews 


Castellini Greta, Gianola Silvia?, Bruschettini Matteo’, Gluud Christian?, Moja Lorenzo* 


1Department of Biomedical Sciences for Health, University of Milan, Italy, 7Unit of Clinical Epidemiology, IRCCS Galeazzi Orthopedic 
Institute, Italy, Department of Research & Development, Section for HTA analysis, Lund, Sweden, ‘Unit of Clinical Epidemiology, IRCCS 
Galeazzi Orthopedic Institute, Italy 


BACKGROUND. Meta-analyses are frequently underpowered. The evaluation of imprecision is an important step in 
developing a systematic review, tempering conclusions when evidence is inconclusive. Imprecision encompasses several 
aspects: sample size, optimal information size, confidence intervals of the overall effect estimate, and pre-specified 
critical margins of ‘no effect’, ‘important benefit’ and ‘important harm’. The Grading of Recommendations Assessment, 
Development and Evaluation (GRADE) combines all aspects in a simple rule, potentially leading to down grade the 
quality of evidence. The Trial Sequential Analysis (TSA) is a frequentist method aimed to reduce the intrinsic uncertainty 
in meta-analyses results, controlling the risk of random error in the context of sparse data. TSA controls the type | and 
type Il errors, combining the calculation of the required information size with the adjustment of the threshold for 
declaring the statistical significance. This approach has been currently suggested as ‘a supplement for a more throughout 
assessment of imprecision’ when using the GRADE system. 


AIMS. To assess the imprecision of the effectiveness of interventions comparing the imprecision assessed by TSA with the 
judgment reported in GRADE summaries in a sample of Cochrane reviews. 


METHODS. Cochrane reviews were screened in reverse chronological order starting on February 2017. A review was 
considered eligible if it (i) was an intervention review, (ii) had a primary dichotomous outcome listed in the summary of 
findings table for the main comparison. One meta-analysis (with at least two trials), for each review, was selected as our 
unit of analysis. Two independent investigators collected data at review and meta-analysis level through a checklist 
created ad-hoc. We extracted the GRADE assessment for the dichotomous outcome and whether it was downgraded for 
imprecision from the summary of findings table. Individual trial data, meta-analytic model and heterogeneity were 
extracted at meta-analysis level. We re-conducted each meta-analysis using individual trial data and applying the TSA 
method (TSA v0.9.5.5 Beta software). We calculated the required information size using a relative risk reduction of 25%, 
unless review’s authors specified a different anticipated intervention effect, an alpha of 0.05, a beta of 0.20. 
Heterogeneity was replicated from the original meta-analysis. Our primary outcome was the agreement of imprecision 
assessment between TSA assessment and GRADE evaluation (downgrade/not downgraded). 


RESULTS. We reported the results obtained from the first 10 SRs out of the 100 SRs expected to be analysed. Among our 
preliminary sample, three out of 10 were new reviews while seven updated. Seven reviews out of 10 downgraded the 
quality of the evidence assessed by GRADE for imprecision: reasons were mostly related to wide confidence intervals 
while only one SR considered the size of the sample. Clinical thresholds were discussed in one review but thresholds were 
not specified. The unweighted Cohen kappa coefficient was 0.53, indicating a moderate agreement between GRADE and 
TSA. In these preliminary results, TSA seemed to be more conservative in judging imprecision, often downgrading by two 
levels, as compared with GRADE, which often downgraded by one level. 


LIMITS. The analyses are restricted to dichotomous outcomes. 


CONCLUSIONS. How uncertainty can be better determined and considered in systematic reviews is still a matter of 
debate. Different methods can support reviewers in understanding how much confidence to place in cumulative overall 
estimates, and communicate the chance of random error. 


Corresponding author: Castellini Greta — University of Milan —- Email: gre.caste@gmail.com 
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12. Implementing GRADE in a Canadian Health Authority 


Chunick Susan’, Hejazi Samar’, Purdon Michelle?, Mick Julie, Vosilla Ann?, Greg Rowell? 


1Fraser Health Authority, Canadian Agency for Drugs and Technologies in Health, 7Vancouver Coastal Health 
Authority 


BACKGROUND. The Fraser Health Authority (FH), one of Canada’s largest health regions serving 1.8 million residents in 
the lower mainland area of British Columbia, provides an integrated service delivery system of 12 acute care hospitals, an 
outpatient care and surgery centre, 7,760 residential care beds, mental health and public health clinics and home and 
community care services. In order to ensure consistent service delivery, clinical decision support tools (CDSTs) are 
developed by inter-professional teams of health care providers and implemented into practice. CDSTs include practice 
guidelines, protocols, procedures, care paths and order sets. In order to ensure that CDSTs are based on a valid analysis 
and synthesis of the published research evidence, FH implemented a ‘Clinical Decision Support Tool Development - 
Clinical Policy’ which requires that the GRADE methodology would be implemented by February 2014 as a means of 
providing a systematic, transparent and rigorous method of evaluating the relevant research evidence in order to make 
recommendations about applying research into practice. The GRADE methodology is now used by inter-professional work 
teams for assessing the evidence used in the development of therapeutic clinical decision support tools ranging from 
clinical practice guidelines to new or newly applied evidence used in pre-printed orders (PPOs). 


AIMS. This presentation will present a case study analysis of the following experiences: « piloting the use of GRADE prior 
to approval as policy, * decisions taken in order to gain executive support, « developing and implementing education 
modules to train inter-professional work teams in the use of the GRADE method for evaluating randomized controlled 
trials and observational studies, * developing a partnership between FH Library Services who develop the PICO question 
with the inter-professional work team and the Canadian Agency for Drugs and Technologies in Health (CADTH) whose 
staff conduct the literature search, * implementing the use of GRADE by the FH Clinical Policy Office, and, » the formative 
evaluation of the use of the GRADE method. 


METHODS. Methods: 1. Using the policy framework of ‘legitimacy, feasibility and support’ and the change management 
framework “ADKAR’” (Awareness, Desire, Knowledge, Ability and Reinforcement), the analysis will identify key enablers for 
change that supported the implementation of GRADE, how professional behaviours were changed, and also will identify 
obstacles that arose during the implementation of the GRADE tool and strategies that were put into place to mitigate 
these. 2. The formative evaluation will include the collection of qualitative feedback from: o the Clinical Policy Office and 
Shared Work Team members about their experience in both following the GRADE process as intended and in using the 
GRADE data collection tool (used to summarize the quality of the evidence, the magnitude of the effect and the reasons 
behind a judgement); o FH Librarians and CADTH library search staff concerning the development of the PICO question 
and use of ‘search’ terms. Quantitative data for the number of CDSTs based on GRADE and therapeutic area will be 
collected from the Clinical Policy Office. 


RESULTS. Results: These will include measures of output, i.e, # of GRADE based CDSTs and outcome measures such as 
increased confidence in the evidence-base for Fraser Health CDSTs in addition to the qualitative data described above. 


LIMITS. Limits: The analysis is not based on a research design. It is evaluative in nature and will focus on a description of 
the experiences in implementing the use of the GRADE methodology as an evidence-based method to integrate the use 
of the best evidence in the development of clinical decision support tools in order to maximize the delivery of quality of 
care. The analysis will not extend to how the CDSTs are implemented in practice once developed. 


CONCLUSIONS. Conclusions: The analysis will conclude with recommendations for how best to implement the GRADE 
methodology in a publicly funded health care system. 
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13. The Cochrane knowledge translation strategy 


Churchill Rachel’, Tovey David?, Champion Chris?, Thompson Denise’, Green Sally* 


University of York, ?Cochrane Central Editorial Unit, ?Cochrane Central Executive Team, “Cochrane Child Health Field, Cochrane 
Australia 


BACKGROUND. Cochrane promotes evidence-informed health decision-making by producing high-quality, relevant, 
accessible systematic reviews and other synthesized research evidence. Cochrane’s vision is a world of improved health 
where decisions about health and health care are informed by high- quality, relevant and up-to-date synthesized 
research evidence. Realization of this vision relies both on production of Cochrane Reviews, and on effective strategies to 
facilitate their use. Knowledge Translation (KT) is at the core of Cochrane’s Strategy to 2020. Goals two and three are 
focused on engagement and meeting the needs of our existing and potential users. Parts of the organization already 
undertake excellent KT, but these activities are not systematic and coordinated across the organization. To properly 
understand what is required to deliver the Strategy to 2020 commitments, we need a strategic view of what should be 
done in KT. The Cochrane KT Strategy is intended to provide that strategic view, and will guide a later implementation 
plan for how we propose to achieve these KT objectives for the organization. 


AIMS. The Cochrane KT Strategy provides clarity around Cochrane’s role in KT and the activities that should be considered 
as priorities for the whole organisation, as well as for individual Cochrane Groups. We want this KT Strategy to facilitate 
significant culture change in Cochrane, so that KT becomes accepted as an integral part of our work. The Cochrane KT 
Strategy should help ensure that KT becomes embedded in all that Cochrane does. 


METHODS. A KT Working Group was convened, involving thought-leaders from within, and external to Cochrane. The 
Working Group was jointly chaired by three Cochrane leads from different parts of the organisation, and was supported 
by a member of the Cochrane Central Executive Team. An extensive process of consultation was undertaken, involving 
input from Cochrane members, as well as in-depth interviews and engagement with external stakeholders. The 
information gathered informed the deliberations of the KT Working Group throughout the strategy development process. 


RESULTS. The Cochrane KT Strategy describes six key themes that map broadly to the goals and objectives of Cochrane’s 
Strategy to 2020. These six themes provide a framework for organising our thinking and activity around KT: i. 
Prioritization and co-production of Cochrane reviews ii. Packaging, push and support to implementation iii. Facilitating 
pull iv. Exchange v. Improving climate vi. Sustainable KT processes. As well as highlighting the key areas of focus for 
Cochrane’s KT work, the Strategy also identifies the major audiences we should be serving: i. consumers and the public ii. 
practitioners iii. policy-makers and healthcare managers iv. researchers and research funders. We have framed the key 
audiences Cochrane needs to reach as the ultimate end users of Cochrane evidence - those making decisions about 
health. In many cases, we will access our audiences through intermediaries such as journalists or guideline developers. 


LIMITS. This KT Strategy is intentionally aspirational. We have set out a vision for KT in Cochrane with the understanding 
that it will take several years to begin to cover all these areas of work, and identifying priority elements of the KT Strategy 
will be important. The implementation of this KT Strategy will also necessarily take place across multiple levels of the 
organization, and will be a complex process requiring careful planning and engagement from all Cochrane Groups. 
Furthermore, there is already considerable KT activity occurring at the Group level and centrally, and it will be important 
not to disrupt this work, but to build on and improve it in partnership with those already engaged in this work. 


CONCLUSIONS. Putting users at the heart of everything we do is at the core of Cochrane’s Strategy to 2020. This KT 
Strategy elaborates on Strategy to 2020's fundamental commitment to the dissemination, use and impact of Cochrane 
evidence. The KT Strategy highlights key areas of focus for Cochrane’s KT work, as well as the major audiences we should 
be serving. It demonstrates the breadth and depth of the activities that would enable us to become a KT- centred 
organization and describes six key themes as a framework for organising our thinking and activity around KT. The 
Cochrane KT Strategy will help us define the scope of Cochrane’s KT efforts, guide work and investment, and, importantly, 
focuses us on the role that each type of Cochrane Group can play in KT. This Strategy will be considered for approval by 
the Cochrane Board during its mid-year meeting in April 2017. It is expected that work towards implementation of the 
final approved Strategy will begin during 2017. 
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14. Use of digital storytelling to teach evidence-based breast imaging to radiography students: a 
case study 


Cowling Cynthia?, Bower Susan? 


1Monash University, Dept of Medical Imaging and Radiation Sciences, *Bower Bird Productions PTY LTD 


BACKGROUND. The sensitive and emotive environment of breast imaging involves an integrative approach encompassing 
the technical and psycho-social, which is challenging to replicate for teachers of student radiographers, whilst 
maintaining an appropriate level of pedagogy. Audio/visual teaching resources are available that share the personal, 
emotive experiences of breast cancer patients or practical radiographic techniques, anatomy and pathology, but none 
than combine all. The challenge was to engage both male and female students within an academic environment at a 
more complex level, incorporating the highest level of technical knowledge and ability with the real life emotions 
frequently encountered in this discipline. 


AIMS. To use digital storytelling of a real life breast imaging patient to link skills, knowledge and theory for best practice 
radiography. 


METHODS. A high quality video production featuring a woman undergoing breast imaging was produced in a digital 
storytelling format, for presentation to third year Medical Imaging students at a major Australian university. It included 
her anxieties and her experiences, interactions with medical and health staff, and relationships with family and friends. 
The video was complemented and integrated with short online visual lectures which incorporated anatomy, pathology, 
technique, communication, special procedures such as biopsies and advances in latest breast imaging, self-review 
opportunities for students, practical hands-on sonography labs using breast phantoms and a formal online final exam. 
The Content Management System (Moodle) was designed such that the student was required to follow through each 
video section sequentially. 


RESULTS. The video illustrated tensions, pressures and anxieties of the patient and how these can be ameliorated through 
best practice techniques. Students were surveyed at conclusion of the module. A strong preference for the design and 
flexibility of the module was revealed. 70% enjoyed the module and were engaged by the story-line. Only 5% found it a 
distraction. Unlike previous years, there was 100% pass rate for this component of the course.Interest was sustained in 
fourth year by three students pursuing research projects in Breast Imaging 


LIMITS. : This designed module has been delivered for the first time in November 2016 to one cohort of mixed gender 
students. 


CONCLUSIONS. Mammography is a sub-speciality of radiography, which is facing a critical workforce shortage in 
Australia. Although there is no shortage of general radiographers at this point, there has been a sustained lack of interest, 
particularly among younger professionals in the discipline of Mammography. As well as meeting the requirements for 
practical experience in the clinical workplace for student radiographers, this innovative pedagogic approach integrating 
knowledge, behaviour and attitudes, created significant interest in the field. This module can also be used as a stand- 
alone module in any program, such as a Medical Degree, Nursing or Breast Imaging training. It can be offered in a fully 
online format. 
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15. Checking the checkboxes: a critical appraisal of clinical quality measures 


Drabkin Alan’, Alper Brian, Qaseem Amir? 


EBSCO, *American College of Physicians 


BACKGROUND. Physicians are increasingly bound by reimbursement contracts which include the use of quality 
performance measures. Compliance with inappropriate quality measure implementation can increase the provision of 
unnecessary medical services with consequent harms to patients, waste of resources, and decreased patient and clinician 
satisfaction. Beginning 1/1/2017, United States physicians who contract with Medicare through the Merit-based Incentive 
Payment System (MIPS) must choose 6 quality measures by which to be evaluated. The evidence basis for these quality 
measures has been questioned. The American College of Physicians has published narrative assessments of some quality 
measures and has found several unsupported by evidence. A systematic approach for the evaluation of appropriateness 
of a quality measure is needed. 


AIMS. 1. To develop a methodology for evidence-based critical appraisal of clinical quality measures 2. To evaluate the 65 
measures in the Primary Care set in MIPS 2017 (including Family Medicine, Internal Medicine and Pediatrics) 


METHODS. .We established 4 threshold-based criteria of pppropriateness to evaluate each performance measure: - 
Convincing evidence that action changes clinical outcomes - Convincing evidence that desirable consequences outweigh 
undesirable consequences -The population is adequately specified for appropriately targeted quality implementation - 
The intervention is adequately specified for appropriately performed implementation We established a system to 
determine the appropriateness of the performance measure according to the results of the Criteria review:. - If all criteria 
were met, performance measure was designated as “Meets Criteria for appropriateness.” - If criteria 1 and/or 2 were not 
met, measure was designated as “Does not meet criteria.” - If criteria 3 and/or 4 were not met, measure was designated as 
“Meets criteria - modification suggested.” We performed a review of relevant literature including PubMed, relevant 
organizational guidelines and existing Dynamed Plus content. We used this methodology to evaluate each of the 65 
Primary Care outpatient quality measures (including Family Medicine, Internal Medicine and Pediatrics) included in MIPS 
for 2017. 


RESULTS. For 65 Primary Care quality measures in MIPS 2017 - Met criteria for appropriateness - 20 measures - Met 
criteria with modification suggested - 24 measures - Did not meet criteria for appropriateness — 21 measures 


LIMITS. 1. The quality measures included in MIPS 2017 are applicable only to physicians in the United States 2 The 
measures included in future years may change according to new policy initiatives. 3. This evaluation does not cover all 
the measures in MIPS, including the remaining 206 measures across 25 specialty groups. 


CONCLUSIONS. 1. Almost 2/3 of MIPS 2017 Primary Care clinical quality measures have inadequate patient-oriented 
evidence to support their use or lack specificity for their implementation. 2. This evidence-based critical appraisal 
analysis of quality measures can - Inform the selection of measures that should be prioritized by clinicians -Influence the 
creation, assessment, and implementation of health care quality measures - Facilitate advocacy for -adoption of more 
meaningful measures likely to induce improvements in health or costs -Retirement of poor measures that may contribute 
to waste and harms 
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16. Pragmatism in evidence synthesis and translation: a perspective on the evaluation of systems 
transformation 


Fowler Davis Sally, Kelly Shona, Hilary Piercy 
Centre for health and Social Care Research, Sheffield Hallam University 


BACKGROUND. Evaluation practices are more or less oriented to the utilisation of the data and conclusions (Patton 2011). 
Utilisation evaluation promotes the careful consideration of feasibility, stakeholder engagement within a framework of ethical 
and respectful boundaries. In a recent evaluation of the Extended Primary Care Programme (EPCP) in a Northern UK City, a 
population-based, health outcomes approach was taken, with the goal of facilitating a formative and summative evaluation of 
benefits achieved through facilitating patient access to primary care. The EPCP was a £9.5million investment in primary care; 
longer opening hours in general practice, additional pharmacy provision and a range of schemes to enable integrated inter- 
professional teams. This paper reports on some of the tensions and methodological problems to be overcome when working 
with a range of stakeholders in complex evaluation and conclusions are drawn from a critical reflection on the process and 
outcomes of this particular programme of work. 


AIMS. This is a critical reflection on the outcomes of an evaluation that sought to support leaders at all levels to make the best decisions 
about systems improvement support by best-evidence (Chauhan et al 2017). The evaluation of the EPCP is used as an example of 
systems transformation requiring complex programme evaluation to report on the health outcomes and impact. 


METHODS. The EPCP evaluation received University ethics approval for a mixed methods design including process evaluation and 
patient and public involvement. The critical reflection is an ‘after action review’ so that organisational learning is captured for further 
systems-academic partnerships and to promote utilisation evaluation planning and impact. 


RESULTS. 1. The 16 schemes of activity were mapped to the outcomes of the whole programme. The mapping and contracting 
activity identified that health outcomes were not available. 2.Scheme level activity data was contracted but the majority of 
schemes were only able to report activity data, based on additional investment in new capacity, with no linkage to effectiveness 
and only one scheme used patient experience survey . 3.The Return on Investment (ROI) calculations were restricted by 
permissions to link to the Hospital Episodes via NHS number (NHS Digital) thereby additional activity couldn't be linked to the 
utilisation of secondary care. 4.The PPI recruitment was achieved but recognised that they were not the users of new services. 
The infrastructure to undertake further engagement and or patient education was not part of the transformation programme. 
5.Four process evaluation events were planned to engage stakeholders. After the first event -that was well attended and 
evaluated well- further events were delayed or cancelled, resulting in a reduction in the opportunity to engage in active and 
multidirectional dialogue about the evaluation outcomes. 6.Methodological limitations and problems were encountered and in 
some cases managed., for example the qualitative data achieved a framework analysis of GP perceptions of service 
transformation and improving access that was valuable and important to report on the critical knowledge that GP's hold about 
demand on primary care 7.The overall evaluation reported on significant additional activity in primary care but was unable to 
report demand management at local level. Additional services/ patient appointments were taken up, demonstrating that 
capacity stimulated or met further demand. 


LIMITS. Data synthesis within evaluation is a particularly 'real-world' academic activity and requires further development and 
methodological development. This critical evaluation presents some of the problematic methodological and stakeholder issues 
encountered. The outcomes of this evaluation fell short of the expectations to fully synthesise the evidence and outcomes of a 
substantial systems transformation programme. We observed a lack of capacity in stakeholders’ understanding of health 
informatics and systems transformation that reduced the impact of the evaluation 


CONCLUSIONS. Complex programme evaluation is an important facet of systems transformation and clinical managers and 
systems leaders are committed to improving services but seldom have experience with utilisation of evaluation. A structured and 
facilitated process included qualitative and quantitative elements, a process evaluation plan to bring partners and stakeholders 
to utilise primary data in planning but there was a need to carefully consider the utilisation context. Patient and public 
involvement (PPI) at programme level was needed to support programme decision-making. Considerable investment and 
expectations of cost reduction and quality improvement were associated with reducing hospital admissions but local quality 
metrics may be helpful too (Barker et al 2017) but this was not achievable. Considerable planning and pragmatic decisions are 
needed to build and correct expectations of data synthesis that supports transformation. 
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17. Teaching the EBP "patient preferences and values" component: which methods and tools are 
validate? 


Guarinoni Milena?, Dignani Lucia’, Sestini Piersante? 


1Universita degli Studi di Brescia, AOU Ospedali Riuniti di Ancona, *Universita di Siena 


BACKGROUND. Evidence-based practice (EBP) has been defined as the integration between the best research evidence 
with clinical expertise and patient values to facilitate clinical decision making. The Sicily statement declares that all 
health care professionals need to understand the principles of EBP, recognize EBP in action, implement evidence-based 
policies, and have a critical attitude to their own practice and to evidence. Accordingly, many health study courses 
introduced the study of Evidence-based practice in their curriculum. These courses usually concentrate on the 
methodology for searching and critically appraising the best evidence available in the literature, and possibly on how to 
adapt the findings to the clinical characteristics of a specific patient and/or setting. However, it remains unclear whether 
there are teaching methodologies and/or tools to facilitate the learning process of the integration of patient preferences 
and values in the EBP process. 


AIMS. In this narrative review we searched the international literature to identify which methods are currently used or 
investigated to teach health professionals how to identify and integrate patient preferences and values in their evidence- 
based practice. 


METHODS. Search methods: We searched the MEDLINE, CINAH, Web of Science, SCOPUS, Cochrane Central Register of 
Controlled Trials and Grey Literature Report up to 4 march 2017. Additional search methods included: screening 
reference lists of relevant studies, and searching websites of selected research groups and organizations. We developed a 


wo wom won 


search strategy for this update with the word "education, professional","patient values", "patient preferences", "patients 
values", "patients preferences”, "Evidence Based" combined in different ways. Additionally, we searched for combinations 
of “Shared decision making” and “Education, professional”, even outside the context of EBP or without explicit reference 
to preferences and values. Selection criteria: We considered all qualitative and quantitative designs research without 
limits of language or time. No limits about health care profession has been placed. Data collection and analysis: Two 
authors independently extracted data and assessed risk of bias. We downloaded all titles and abstracts retrieved by the 
electronic searches into the reference management database EndNote and removed duplicates. One review author 
screened all titles identified by the main search, excluding all studies which clearly did not meet the inclusion criteria. We 
produced a long-list of titles and abstracts and two review authors screened this independently. We obtained the full text 
of potentially relevant papers. We resolved disagreements by discussion between authors or if needed arbitration by a 
third person. 


RESULTS. In the main search, keywords otherwise combined returned 19 articles, none of which did show the contents 
object of the present review. Grey Literature Report did not produce any result. The additional search returned a few 
studies about teaching shared decision making in a variety of professional contexts, only one of which was apparently 
related to a EBP course. The few methods which emerged, involved the extensive use of standardized patients and 
videotaping, which are beyond the possibilities of most educational settings. 


LIMITS. Our inquiry was limited to published research. It doesn't covers methods which could be currently under testing, 
or unpublished experiences. 


CONCLUSIONS. Although evidence based practice should be based on the integration between the best research 
evidence with clinical expertise and patient preferences and values, there is little research about methods and tools to 
teach how to integrate the latter in the decision process, mostly outside the context of EBP education. New research is 
therefore needed to develop, and embed in the EBP curriculum, methods and tools to teach and evaluate the skills 
needed to identify, assess and integrate patient preferences and values in the EBP process. 
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18. What factors influence general practitioners’ competencies and application of EBM? 


llic Dragan 


Monash University 


BACKGROUND. General practitioners (GPs) face several challenges when attempting to implement the principles of 
evidence-based medicine (EBM) in practice. Much of the literature to date has focused on the levels of competency and 
organisational barriers and enablers to its implementation. Evidence from a variety of observational, and to a lesser 
extent experimental studies, has demonstrated that any form of educational intervention will increase GP knowledge in 
EBM. Quantitative and qualitative studies have identified accessibility to resources, attitudes toward EBM, lack of 
knowledge and skills, and a consistent evidence base in general practice as significant barriers to the uptake of EBM by 
GPs. Whilst strategies to overcome those identified barriers have been explored, few studies have investigated what 
impact demographics, beliefs, confidence and levels of EBM implementation may impact upon GP competency in EBM. 


AIMS. This project aimed to identify the level of practitioner competency in EBM in a group of current practising GPs. A 
secondary aim was to identify what influence practitioner age, confidence, beliefs and implementation had upon 
competency levels. Barriers and enablers to uptake of EBM in practice was also explored. 


METHODS. A mixed methods study consisting of a quantitative questionnaire and qualitative focus groups was performed 
with GPs. Competency in EBM was evaluated using the Assessing Competency in EBM (ACE) tool. The EBP Implementation 
and Beliefs (IBPIS) and EBP Confidence (EPIC) tools were used to evaluate participants’ beliefs, implementation and 
confidence in EBP. Descriptive statistics were used to summarise data, with Pearson’s correlation and student t-tests used 
to examine relationships between outcomes. Focus groups were conducted by the same facilitator and guided by a semi- 
structured interview schedule. Focus groups were conducted until theoretical saturation, and analysed independently by 
two researchers using thematic analysis. 


RESULTS. In total, 43 GPs with a mean age of 58.1 (+11.4SD) participated in the study. Participants were on average 
identified as exhibiting a novice level of competency in EBM. A moderate, although statistically significant, positive 
correlation between level of competency and beliefs toward EBM was identified (r=0.45). No correlations were identified 
between level of competency and implementation or confidence. A weak, negative correlation was identified between 
competency and age (r= -0.15). Several themes were identified with the discussions with GPs. Lack of formal education in 
EBM was identified as the major barrier preventing GPs from implementing principles of EBM in clinical practice. GPs also 
questioned the need for high level competency in EBM, rather relying on the availability of synthesised evidence such as 
clinical practice guidelines. Experience across all three facets of EBM was identified as a key driver to implementing EBM 
in practice — experience in having a broad depth of clinical knowledge, experience relating to continued practice of 
identifying and appraising evidence, and experience relating to communicating effectively with patients. Participants 
were also of the view that level of EBM competency was positively correlated with beliefs, confidence and experience. 


LIMITS. GPs participating in this study were recruited from one major metropolitan city in Australia. The potential for self- 
selection bias, and the relatively small sample size, limits the generalisation of study results. 


CONCLUSIONS. Level of EBM competency in mid to senior career GPs is multifaceted. Previous formal teaching in EBM 
plays a key role in GPs’ potential competency levels, beliefs and approaches to implementing EBM in practice. Continuing 
professional programs, which employ a combination of group and self-directed learning techniques, along with face-to- 
face and online teaching modes, is required to upskill a large population of GPs who have not been formally educated in 
EBM. 
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19. Appropriateness of eye care: can we use practice records, practitioners’ and patients’ 
perspective to design, target and measure, intervention strategies? 


Jalbert Isabellet, Ho Kam Chun}, Rahardjo Dian’, Gopinath Bamini?, Yashadhana Ary?, Wiles Louise?, Hibbert 
Peter?, Liew Gerald?, White Andrew?, Stapleton Fiona? 


1UNSW Sydney, *University of Sydney, 7Macquarie University 


BACKGROUND. Eye care accounts for a significant proportion of total health care. Limited information is available on the 
appropriateness of eye care in Australia. Existing evidence suggests that compliance to guidelines is sub-optimal in many 
areas of eye care. 


AIMS. To investigate the appropriateness of eye care delivery and explore eye care practitioners and patients’ 
perspectives on barriers and enablers to appropriate eye care in Australia. 


METHODS. Mixed method approach using focus groups, semi-structured interviews and Delphi consensus process for the 
qualitative aspects of the research in combination with nominal group technique, records audit, and Knowledge, Attitude 
and Practice (KAP) surveys for the semi-quantitative and quantitative aspects of the research. Eye conditions glaucoma, 
preventative care, diabetic eye disease, and macular degeneration were selected on the basis of their significant 
contribution to vision loss in Australia. The CareTrack method was used to develop indicators of appropriate care using a 
Delphi consensus process, where established existing evidence (e.g. national and/or international guidelines and 
scientific literature) were translated into indicators of care. Key Australian stakeholders (optometrists, ophthalmologists, 
patients and their carers) were identified and recruited from three Australian states and their views sought. 


RESULTS. Pilot studies have demonstrated the feasibility of review of clinical records from optometry, ophthalmology 
private practices and university-based eye health clinics, to assess appropriateness of eye care delivery. To date, overall 
appropriateness of preventative eye care (69%; Cl 67% to 70%) is in line with overall Australian health care. The diverse 
nature of eye care practices and their clinical record platforms (i.e. electronic record, structured, and unstructured paper 
records) present many challenges. Illegibility and usage of nonstandard abbreviation, and missing data complicate data 
collection. Numerous (50+) barriers to eye care were identified by patients and their eye care practitioners in the area of 
macular degeneration care. Complex factors such as lack of funding, the importance of a defined multi-disciplinary care 
approach and the challenges involved with dietary behaviour changes were identified as key priority areas. Patients 
generally demonstrated poor understanding of their eye disease and treatment and this could be contrasted with eye 
care practitioners’ generally positive comments regarding the quality and clarity of the doctor-patient conversation. 


LIMITS. Clinical records are unlikely to accurately document and reflect real life practice. Patients and practitioners with 
very negative views or those that perceive themselves as delivering lower quality eye care may not have been willing to 
participate in the research and our findings may therefore not reflect their views. While we sampled until data saturation 
was achieved, it is possible that we missed an important patient or clinician perspective. 


CONCLUSIONS. This research involving clinicians and patients seeks to measure and identify gaps in the delivery of eye 
care in Australia and use these findings to set priorities for improving eye care delivery in the future. Eye care appears to 
be delivered appropriately for the most part. Increased advocacy for funding, the design and dissemination of care 
pathways involving all stakeholders and interventions to support change behaviours are necessary to improve the 
delivery of eye care in Australia. 
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20. Best practice statements in public health guidelines at WHO 


L Norris Susan, Ferri Mauricio, lvey Sawin Veronica 


World Health Organisation 


BACKGROUND. Best practice statements (BPS) may be provided in guidelines in lieu of evidence-based recommendations 
when there is a high level of certainty that the benefits of the recommended intervention outweigh the harms. However, 
BPS are not clearly defined and terminology is inconsistent, leading to both overuse (an evidence review and standard 
recommendation should have been developed) and underuse (performing evidence reviews when none is necessary). 


AIMS. To describe BPS in WHO guidelines and to propose a new definition and typology for BPS. 


METHODS. All guidelines approved by the World Health Organization (WHO) Guidelines Review Committee and 
published from 2012 to 2016 were reviewed. We included discrete statements issued by guideline expert panels that were 
not based on a review of evidence. We extracted the characteristics of these statements and synthesized data using 
descriptive statistics. An iterative, consensus-based process was used to formulate definitions and a typology for BPS 
based on this cohort of guidelines. 


RESULTS. Of 86 guidelines, 31 contained BPS. These statements were variably labelled and presented, and the process by 
which they were developed and their rationale were often poorly reported. Several discrete types of BPS emerged, 
including those based on: 1) human rights and ethics principles and conventions; 2) indirect evidence based on physics or 
other principles; 3) indirect evidence based on established clinical principles; 4) the need to collect information; and 5) 
other reasons where the BPS does not reasonably require the systematic collection of evidence. 


LIMITS. This study only assessed discrete statements. We did not review the guideline text in full to identify BPSs 
potentially imbedded in the text. 


CONCLUSIONS. This work may help guideline developers consider different types of BPS and to avoid their inappropriate 
use. The typology needs to be tested in different guideline cohorts for reliability and utility, and as a tool in the 
development of future guidelines. 
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21. Technology workarounds and patient safety 


Mastrian Kathleen 


Penn State University 


BACKGROUND. Healthcare technologies are designed and implemented to improve the work of managing and delivering 
healthcare in increasingly complex and fast-paced environments. Many of these technologies are focused on ensuring 
patient safety. However, the implementation of new technologies and safety strategies may disrupt the typical workflow 
of clinicians. In the interest of meeting workload demands, clinicians may use a technology workaround that has the 
potential to compromise patient safety. Technology workarounds are deviations from accepted and expected practice 
protocols. At times, the organizational culture may actually promote these risky behaviors. It is imperative that we 
understand the nature of these behaviors and develop strategies for addressing the technology workflow disruption and 
the human factors engineering principles to promote a safety culture. 


AIMS. Explore the characteristics of a safety culture, and strategies for promoting a culture that avoids risky behaviors. 
Introduce cognitive informatics principles and how they influence human factors engineering and technology usability. 
Define technology workarounds. Examine case reports related to technology workarounds. Propose strategies for 
assessing the characteristics of technology workflow disruptions and preventing technology workarounds. 


METHODS. Qualitative data on technology workarounds will be presented from both current literature and practicing RN 
reports of actual use or observation of technology workarounds in clinical settings. 


RESULTS. The most common technology workarounds are related to medication administration. More specifically these 
workarounds typically involve patient ID workarounds, bypassing smart pump technologies, and bypassing second 
clinician verifications for high-hazard medications. A second category for technology workarounds is bypassing or 
ignoring alarm systems. Also reported is the inappropriate use of EHR functions to complete patient documentation. 


LIMITS. The actual case reports from practicing RNs is small in size (N=25). In addition, the examples are primarily from 
the US. 


CONCLUSIONS. As technologies that can help to reduce errors and increase safety are increasingly integrated into 
caregiving activities, healthcare professionals must commit to using the technology in the way that it was designed. The 
technology must be scrutinized and tested routinely to ensure that it is not disrupting clinical workflow. Users must be 
adequately educated so that they use the tools appropriately to ensure patient safety. Finally, the organizational culture 
must embrace the ideas, opinions and strategies proposed by end users to improve the human technology interface. 
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22. Adding value and reducing research waste: the role of the research management community 


Minogue Virginia 
NHS England 


BACKGROUND. The series of papers published in the Lancet in 2009, 2014, 2016 identified a number of stakeholders 
(funders, sponsors, regulators, researchers) with a role in reducing research waste. The Research management community 
(Directors, Managers and support staff) -- were not seen as playing a key role but a project based in the English NHS 
found that this community were motivated and engaged and able to identify and address research waste. The Close 
Partnering work stream of the NHS Future Focused Finance programme identified reducing waste as a key area of work. 
Research is one area identified as contributing to the high percentage of NHS expenditure that is wasted (Minogue and 
Wells 2016). Eighty five per cent of research is described as avoidably wasted or ineffective (Chalmers and Glasziou 2009) 
and fifty per cent of waste is due to gaps between research and practice (Cartabellotta 2015). Studies have shown that 
identifying and eliminating waste can lead to better prioritisation of research, improve the yield from research and make 
it more focused on clinically relevant issues. A failure to get research into practice results in little impact from studies. 
Research is crucial to our understanding of what is high value care and provides evidence that is vital for decisions of 
value and to support disinvestment and reallocation of funding. This Research community have a key role to play and this 
project is aimed at building their understanding and knowledge about the role they can play in reducing waste. 


AIMS. To demonstrate to NHS research, development and innovation staff that they have a role in considerations about 
reducing waste and the delivery of high value care. To empower NHS research and development managers and leaders 
to use the opportunities they have to influence decisions within the research pathway. Ensure research and development 
managers and leaders contribute to the delivery of better value and high quality research which delivers real impact for 
patients. To support the implementation of the REWARD recommendations and the NIHR ‘Adding Value in Research’ 
Framework. 


METHODS. The project involved four stages: Stage 1 - mapping of the research community and influences; Stage 2 - 
consultation with research managers to identify and agree categories of research waste (based on REWARD and Lancet 
papers 2009, 2014, 2016). Mapping of data against the NIHR Adding Value in Research framework. Stage 3 — further 
consultation with the research manager community to identify areas of improvement and barriers. Further consultation 
with other stakeholders including the NIHR. Stage 4 — dissemination of findings. Development of framework to support 
research managers in addressing research waste. An expert group of patients provided critical review of the findings. 


RESULTS. Identification of the areas of research waste that research managers regard as the most important and have 
most value to the research process if addressed. Identification of research manager’s spheres of influence and the 
barriers to tackling research waste. Engagement of the research management community in reducing research waste. A 
framework to support research managers in addressing waste. 


LIMITS. The research management community's ability to influence those areas of research waste they see as most 
important i.e. prioritisation and implementation of research. 


CONCLUSIONS. Research managers need to work with the rest of the research community to identify and address 
research waste and to use their influence to address specific areas of waste. 
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23. Rapid reviews of research: a tool for supporting evidence-informed health policy 


Moore Gabriel, Redman Sally, Rudge Sian, Haynes Abby 


The Sax Institute 


BACKGROUND. Rapid reviews are tailored, targeted syntheses of research that answer specific policy questions arising in 
‘real world’ policy and program environments.They give policymakers access to the best available research in short 
timeframes. As such, they are an essential part of the evidence-for-policy ecosystem. Despite the growing number of 
programs, little is known about how rapid reviews are used by health policy makers. 


AIMS. This study examined whether and how rapid reviews commissioned using the Evidence Check rapid synthesis 
program were used by Australian policymakers. 


METHODS. We used a systematic interview process to examine the use of 139 rapid reviews that were commissioned by 
health policy agencies between 2006 and 2015. We developed a coding schedule using deductive and inductive 
processes, and used Fisher’s Exact Test to assess variation in use. We examined how rapid reviews were used, when they 
were used, what evidence of use was provided and what reasons were given when the reviews were not used. 


RESULTS. Eighty-nine percent of rapid reviews were used by the commissioning agencies, with 338 separate instances of 
use identified. Policymakers used reviews primarily to: determine the details of a policy or program; identify priorities for 
future action or investment; negotiate decisions across jurisdictions; evaluate alternative solutions to a policy problem; 
and communicate data and strategic information to stakeholders. Some variation in use was observed across agencies. 
Reasons for non-use related to changes in structures, resources or key personnel. Reviews were used mostly in 
instrumental and conceptual ways, and there was little evidence of symbolic use. Variations in use were identified across 
agencies. 


LIMITS. It is possible that the 11 reviews for which we do not have systematic follow-up may have introduced a bias 
towards reviews without interviews being less likely to be used. It is also possible that social desirability influenced 
respondents to report more instances of use than occurred; however we feel that the high level of specificity given about 
reported use minimised this likelihood. Lastly, the findings refer to one rapid review program (Evidence Check) and the 
results may not be generalisable to other rapid review programs. 


CONCLUSIONS. We found that almost all rapid reviews had been used by the agencies who commissioned them and were 
used in multiple and diverse ways. The study identified for the first time that rapid reviews were commissioned to set 
research agendas and inform programs of research.The findings suggest that commissioned rapid reviews are an 
effective means of providing timely relevant research for use in policy processes, and that reviews are used for a variety 
of purposes. 
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24. Do curriculum changes affect EBP competencies of speech-language therapy students? 


Neijenhuis Karin, Verhoef Joan, Dekker- Van Doorn Connie 


Rotterdam University of Applied Sciences 


BACKGROUND. Education in evidence-based practice has become more integrated in the curriculum, as there has been a 
movement from offering separate EBP courses towards integrating an EBP-attitude across all courses in a curriculum. The 
school of health care studies at Rotterdam University of Applied Sciences, including the speech-language therapy 
program, implemented a more integrated approach since 2013. According to the first baseline measurements of EBP 
competency in students in 2013, it was expected that especially attitude and critical appraisal skills still could be 
strengthened. By creating more focus on EBP in daily clinical practice, sustainability of EBP competency should improve. 
Now, after four years, the first students that followed the more integrated programme, will graduate. Their results are 
compared to the results from their peers who followed the former curriculum. 


AIMS. This study aims to compare EBP performance of nearly graduated SLT students before and after a curriculum 
change. 


METHODS. Subjects are two cohorts of fourth-year students speech language therapy who are nearly graduated. In order 
to test EBP performance (knowledge and skills), the Dutch Modified Fresno test is used (Spek et al., 2012). Subjects also 
completed a questionnaire on motivational beliefs of EBP (Spek et al., 2013). 


RESULTS. At the time of the conference, results will be available of two cohorts of students speech-language therapy; 
n=27 for the first cohort and a comparable number of students for the second cohort. Their scores on the Fresno test (EBP 
performance) and questionnaire for motivational beliefs will be statistically tested on differences between groups. These 
differences will be explained with regard to the curriculum changes. 


LIMITS. Although the Fresno test is widely used, there is no gold standard for the level of EBP performance the students 
should attain. Furthermore, the current measurements cannot fully be generalised to real EBP in clinical practice. 
However, we expect that curriculum changes could influence results on these measures. 


CONCLUSIONS. In order to review EBP competency in the curriculum, measuring EBP knowledge and skills as well as 
motivational beliefs is important. It is also important to determine what knowledge and skills are relevant for speech- 
language therapists. Regarding development of curricula: there are different ways in which EBP competencies can be 
taught in the curriculum, but these don’t necessarily lead to a different outcome. The Fresno test, combined with the 
questionnaire on motivational beliefs could be useful instruments to evaluate if an upgraded curriculum leads to more 
competent evidence-based practitioners. 
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25. Forecasting science: a strategy for planning research resources 


Oliver Kathleen 


Kathleen Burr Oliver Consultant 


BACKGROUND. From a recent information technology (IT) services planning effort in the U.S. National Institutes of Health 
(NIH) Division of Library Services (DLS) emerged a suggestion for anticipating needs of experts in industries that 
experience rapid and dynamic change. The strategy consists of asking experts to talk about the future in their fields, and 
then asking them to identify the resources and services that will enable that future. The Office of Research Services (ORS) 
Scientific Resource (SR) Divisions of the NIH, led by the Division of Library Services, saw such an exercise, in this case with 
scientists, as a useful way to inform advanced, strategic planning for resources and services to support the NIH Intramural 
Research Program (IRP). ORS SR divisions include, in addition to library services, instrumentation, equipment, veterinary 
resources, arts and photography, as well as radiation and occupational safety. The NIH IRP 2017 Long Range Planning 
effort identified six research areas for its 2017 planning retreat: inflammatory diseases, genotyping and phenotyping, cell- 
based therapies, neuroscience -—compulsive disorders, RNA biology and therapeutics, and the microbiome. The 
forecasting science study is anchored by these NIH IRP research topics, and, within those, seeks to balance clinical and 
laboratory research perspectives, stratified by those of early and establish career scientists. In the fall of 2016 through 
spring 2017, the ORS began to lay the foundation for a study using such an approach. It explored effective study design 
and engaged NIH leadership to gain support for the recruitment and participation of NIH scientists in the study. In late 
spring 2017, a study plan and timeline was finalized with data collection in place for summer 2017. This abstract will 
describe planning effort and study design, updated at EBHC conference with results and their implications for ORS 
resource planning. 


AIMS. The primary aim of the study is to anticipate changes in the resources and service needs of scientists at NIH in 
alignment with the long range plan for the IRP. 


METHODS. Using a combination of qualitative and quantitative methods, we will collect data from both early career and 
senior scientists who conduct clinical and laboratory-based research in the six research focus areas identified by the NIH 
IRP leadership in 2017. The study includes 6 focus groups with up to 8 participants each for early career scientists. Each 
group is designed to address one of the scientific areas identified in the IRP planning retreat. Laboratory and clinical 
scientists are to be mixed in each group as appropriate. Up to 10 senior scientists will be interviewed as a complement to 
early career scientists focus groups. Informed by findings from focus groups and interviews, a survey of 5200 trainees, 
tenure-track and tenured scientists of the IRP will follow. Both focus group and survey participants will be volunteers. 


RESULTS. We anticipate the data will suggest new services and resources, areas of collaboration with NIH Institute core 
resources, and help validate and prioritize existing or emerging services. The presentation will include detailed analysis 
of the results and describe early stages of their impact on policy, budgeting and planning for resources and services 
designed to support and accelerate scientific research at NIH. Demographic data will enable analysis and presentation of 
results by NIH Institute or Center, as well as other useful perspectives such as research focus, and stage of career. We 
anticipate the results will also facilitate collaboration with IRP leadership in NIH IRP long range planning efforts. 


LIMITS. Because the participants are volunteers, selection bias could affect the generalizability of results. 


CONCLUSIONS. We will test the hypothesis that engaging scientists in discussion of future trends in and required 
resources for their research is an effective tool for planning. We will learn if engaging and coordinating with the 
leadership around NIH planning efforts enabled access to the target population for participant recruitment through 
restricted email lists, and affected their participation as champions for the study. In addition, we hope to learn if 
preliminary leadership engagement built a foundation for collaboration in post-study planning and funding. 
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26. User experience and performance of a mobile application for translating EBP to clinical 
education: the EBPsteps 


Olsen Nina Rydland?, Titlestad Kristine Berg’, Johnson Susanne Gradem?, Heldal Ilona?, Hole Grete Oline?, 
Larun Lillebeth? 


Western Norway University of Applied Sciences, 7Norwegian Institute of Public Health 


BACKGROUND. Students within health- and social care programs lack evidence-based practice (EBP) competence. In 
particularly, they struggle to use EBP and transferring evidence to clinical situations. Mobile-learning technology has the 
potential to become a central tool for facilitating learning and use of EBP. To better equip students to meet the 
expectations of making their clinical practice evidence-based a mobile application (app) called “EBPSteps” was 
developed at the Western Norway University of Applied Sciences. The EBPSteps app guides users through the five EBP 
steps (ask, search, appraise, integrate and evaluate), enables documentation of the process, and provides users with links 
to a Norwegian internet-based learning resource in EBP. Before implementing this new methodology, we need to 
increase our understanding of the usefulness of the EBPSteps in clinical education, in comparison with existing ways of 
encountering EBP 


AIMS. To explore user experience and performance of the EBPsteps among bachelor students who have used the app in 
clinical education 


METHODS. Focus group interviews were conducted in February 2017: one with four 3rd year social educator students and 
one with three 3rd year occupational therapy students. Further, three focus group interviews are planned in April/May 
2017, with different student cohorts. Results from these five interviews will guide further improvement and development 
of EBPsteps. When these improvements are completed in autumn 2017, we will invite student cohorts in clinical 
education to pilot the EBPsteps and to participate in a second round of focus group interviews to enable further 
improvements. Interviewing different participant-categories will ensure comparative analysis and enable us to exploit 
differences in perspectives and interactions. Interpretive description will guide the data collection and analysis. 


RESULTS. Three integrative themes emerged from the analysis from the two focus group interviews: “the need for EBP 
competence’, “relevance for EBP” and “good and intuitive user interface”. The uptake of the EBPsteps seemed dependent 
on whether the students experienced the app as relevant and useful. For example, they used the app when they 
experienced information needs and when the school or their supervisor required or challenged them to use the app. 
Whether they had positive experiences using the app dependent on their previous competence in EBP and familiarity 
with the EBP steps. In particularly, lack of searching skills for research evidence was identified a barrier towards following 
all the EBP steps. Links to learning resources integrated in the app were helpful where competence was lacking. Students 
preferred links in the app to books, and checklists for critical appraisal in the app to paper versions. Students experienced 
the interface as intuitive, as the app gave a good overview of the EBP process, facilitated the process with following the 
EBP steps and enabled them to store information in one place. Students experienced that some of the EBP steps were 
easier to do on a computer than a phone, e.g. searching in databases. However, not all students realised the potential of 
the app, for example, that they could use the app on both their phone and their computer; the possibility to send 
information from the app via email or that there was a glossary in the app. Our findings show that there is a need for 
developing an instruction video and we need to facilitate opportunities for practicing use of the app before students start 
clinical education. 


LIMITS. Further systematic piloting of the tool is needed before extensive implementation. 


CONCLUSIONS. We found that uptake of the app among students dependent on their EBP competence, if they perceived 
EBP as relevant, and a good and intuitive user interface of the app. The EBPsteps is a promising tool for translating EBP 


into clinical education within health- and social care programs. 
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27. From evidence to practice: reflections on collaborating and co-producing with the Sensory Trust 


Orr Noreen, Maguire Kath, Whear Rebecca, Thompson-Coon Jo, Briscoe Simon, Abbott Rebecca, Garside Ruth 
University of Exeter 


BACKGROUND. The Sensory Trust charity, based in the South West of England, delivers a range of projects promoting 
sensory approaches to the natural environment in the belief that they bring social, health and well-being benefits. Its 
confidence in the value of sensory approaches for people often experiencing social exclusion has been largely based ona 
deep tacit knowledge developed from over 20 year’s experience of practice and project evaluations. However, the 
recognition of the need for a more established evidence base for its approaches led to a collaborative research initiative 
with the University of Exeter. 


AIMS. To present and reflect on the experience of collaborating and co-producing with a charity on how we worked 
together developing the research question, undertaking the evidence synthesis, disseminating the findings and designing 
a training toolkit for delivery in the residential/nursing care sector. 


METHODS. Collaboration between the Sensory Trust and the University of Exeter began in 2013 with a three month in- 
residence scheme. This enabled both to explore the potential research opportunities around a theme of mutual interest - 
wellbeing and the natural environment. The University hosted a workshop for the Sensory Trust on the process of 
systematic reviews and on how they could be used to explore the evidence base for sensory approaches, and the Sensory 
Trust shared its knowledge on sensory approaches and the various contexts in which they are applied. The outcome of 
this residency was the identification of a research question: * When using natural settings, how do older people describe 
their sensory engagement with the outside world? o Are there different experiences for different groups of people (e.g. 
those with dementia)? o Are there ways in which these experiences can be enhanced? With follow-on-funding from the 
University, we embarked on a six month collaborative project whereby a researcher from the Sensory Trust was seconded 
to the University research team to synthesise the qualitative evidence on how older people describe their sensory 
engagement with the outside world. This enabled us to combine the University’s in depth knowledge of qualitative 
research and systematic review methodology with the specialist knowledge of the Sensory Trust. The co-produced 
evidence synthesis (Orr et al, 2016) found that: » older people’s sensory descriptions centred on the visual dimension of 
their sensory engagement and their multisensory experiences of ‘being’ and ‘doing’ in the natural environment; « the 
descriptions offered by older people stressed the pleasure and enjoyment they had from connecting with the natural 
environment. However, the evidence also found that some older people and particularly those living in 
residential/nursing care can experience barriers to accessing the outdoors. These review findings empowered the Sensory 
Trust to advocate for changes in practice amongst care professionals and, with further funding from the University, we 
were able to hold a workshop in May 2016 to share the research findings with representatives from the care sector. At 
this workshop the Sensory Trust acted as a connector and we had an opportunity to find out how the findings could be 
more user-friendly for the care sector. 


RESULTS. We have now reached the implementation stage in our collaborative journey with the Sensory Trust, and a 
Project Co-creation Grant is enabling us to implement our review findings. Our current work aims to enhance the quality 
of life for older people living in residential/nursing care (includes older residents with dementia) by collaborating with 
the Sensory Trust to develop ‘My Nature: a Training Toolkit’ for delivery in the residential/nursing care sector. The Sensory 
Trust is leading on designing the training toolkit to enhance older care home residents’ experiences of the natural 
environment. The training toolkit will be piloted in two care homes in the South West of England, to co-produce case 
studies for regional and national distribution to stakeholders and organisations involved in the care of older people. 
Piloting is vital in understanding the context in which the training toolkit will be used and how best it can be 
implemented as part of routine care. 


LIMITS. The benefits of sustained engagement that we have experienced maybe specific to this charity and/or area of 
work, but we will look to mirror this collaborative model where we can in future research. 


CONCLUSIONS. Our collaboration with the Sensory Trust has been ongoing since 2013 and over time we have been able 
to build trust which has fostered a willingness to continue our working relationship beyond the publication of the 
research findings. The collaboration illustrates ways of involving a partner in all parts of the research process from the 
shaping of the research question, involvement in doing the review, interpretation of the findings, and dissemination and 
implementation of the results. Engaging with the Sensory Trust in active collaboration has resulted in a sustainable and 
rewarding way of working together. 
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28. The development of “Evidence into Practice — Rapid Reviews” 


O'toole Eve, Heckmann Patricia 


National Cancer Control Programme Ireland 


BACKGROUND. The National Cancer Control Programme Ireland (NCCP) has developed a robust evidence based 
methodology for developing clinical guidelines. One of the ongoing challenges that guideline groups face it responding 
to rapidly changing evidence as guidelines can take two years to develop. Another global constraint on health services is 
pressure on health budgets. One of the big contributors to the problem is the escalating cost of oncology drugs which can 
cost upwards of €100,000 per QALY. 


AIMS. This rapid-review addresses the need of quick guidance along with providing the payer with reassurance around 
the impact of these treatments 


METHODS. An Evidence into Practice Rapid Review process was developed. This is modelled on the guideline 
development process but aims to address a limited number of clinical questions, in a short time period, on areas with new 
and emerging evidence, where there is variation in practice and potential to have large impact on patient outcomes. This 
was trialled on new immunotherapy drugs developed to treat metastatic melanoma. This is clinically significant as these 
patients typically have less than one year life expectancy. The following steps are carried out: * A team is established 
consisting of four consultants,a pharmacist, a researcher, a librarian and methodology lead. * Clinical questions are 
developed « Literature searches are performed « High level evidence is is extracted into data-tables » A face-to-face 
recommendation meeting is held. Short evidence statements and clear recommendations on the use of the drugs are 
written. 


RESULTS. The rapid review is submitted to the drugs group in conjunction with the NCCP recommendation on funding. It 
provides clinicians with treatment algorithms in areas of new and emerging evidence. There is a predetermined time 
frame for updating recommendations as new evidence continues to be published. 


LIMITS. One of the main limititations is accessing a quorum of expert cliniians, in a timly manner, when they are in a 
system with clinical emergenices under-resourced and without sufficiant numbers. 


CONCLUSIONS. This innovation enables the development of rapid guidance and ensures, emerging evidence can be put 
directly into practice to improve patient outcomes while providing assurance about budget impact. By reducing variation 
in practice we can monitor real would outcomes in the Irish setting and contribute to the growing evidence base on these 
topics. 
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29. Applicability of research to rehabilitation clinicians: a qualitative study 


Patel Drasti?, Koehmstedt Christine, Gerber Lynn’, Jones Rebecca’, Coffey Nathan’, Cai Cindy’, Garfinkel 
Steven’, Shaewitz Dahlia?, Weinstein Ali? 


1George Mason University, George Mason University, 7Inova Fairfax Hospital, George Mason University, George Mason University, 
3American Institutes for Research, American Institutes for Research, American Institutes for Research, George Mason University 


BACKGROUND. There is limited research examining whether rehabilitation specialists feel that the published literature is 
relevant and useful to inform practice. While there has been a growing amount of literature examining knowledge 
utilization and research preferences of physical therapists (PTs) and occupational therapists (OTs), research examining 
the information seeking behavior and implementation of research findings of other rehabilitation clinicians is still sparse. 


AIMS. The objective of this study was to examine whether rehabilitative clinicians including PT, OTs, rehabilitation 
counselors (RCs), and physiatrists perceived the current research literature as relevant to inform their clinical practices. 


METHODS. Twenty-one clinicians were administered semi-structured interviews via telephone. The audio recordings of 
the interviews were thematically analyzed using NVivo. Prior to performing the analysis on participants’ responses, the 
research team developed a bank of answers and themes. The answer and theme codebook was created based on 
responses given to open-ended questions in a random sample of the interviews that included a combination of all 
clinician types. Then all of the interviews were coded using the developed codebook. 


RESULTS. There were similarities found with respect to the information seeking behaviors and approach to the 
translation of research across the different clinician types. Lack of time was reported to be a barrier for both access to 
and implementation of research across all clinician types. Clinicians noted that a supportive work environment was 
helpful in gaining access to literature for their clinical field. The majority of clinicians that reported having difficulty 
utilizing the published literature indicated that the literature was not applicable to their practice, they could not find a 
relevant research study that mapped directly to the patient that they were trying to treat or questions that they were 
trying to answer. Other reasons cited for the inapplicability of the research literature was that the research protocols 
were not feasibly designed for clinical practice. For example, the amount of time of the research protocols (i.e., 2-3 hours 
of therapy when the clinician only gets an hour with the patient) or the equipment used was not accessible to the 
clinicians (i.e., the current focus on robotics when it is not accessible to many clinicians). 


LIMITS. A limitation of this study was that male physiatrists and female RCs, PTs, and OTs participated. Therefore, 
perspectives from female physiatrists and male RCs, PTs, and OTs were not included. In addition, all of the clinicians 
interviewed were a part of clinical practices that had at least 6-10 practitioners, so smaller clinical practices were not a 
part of this sample. 


CONCLUSIONS. Our study identified several barriers that affect rehabilitation clinicians’ ability to utilize the existing 
literature in their clinical practice. These barriers predominantly included time constraints, as well as problems with the 
applicability of available literature. Our findings suggest the need for researchers to communicate with consumers of 
their research: patients, caregivers and professional providers before commencing their research. This will increase the 
likelihood of research applicability. 
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30. What kinds of knowledge types are constitutive for knowledge translation? Some 
epistemological remarks on EBHC 


Reiersgaard Anders 


Western Norway University of Applied Sciences 


BACKGROUND. The essence of Evidence-Based Health Care (EBHC) is to use research evidence by translating it into 
practice. What does this translation entail, what types of knowledge are necessary to constitute such a translation, and 
how is such knowledge to be integrated in EBHC education? Answers to these questions require clarification of what 
kinds of knowledge that is constitutive of EBHC in general, and of how the different kinds of knowledge are related. Both 
scientific and practical knowledge play important roles in EBHC, but the role of the latter is rather unexplored in EBHC. 
Practical knowledge is typically expressed through more specific knowledge-types, such as “clinical reasoning”, “clinical 
expertise”, and “integration”. These knowledge-types have been addressed by both proponents and opponents of EBHC, 
but have not been dealt with in sufficient detail. Thus, it remains unclear what the role of practical knowledge is and how 
it is to be integrated as a feature in EBHC in general, and in evidence-translation in particular. 


AIMS. To investigate knowledge types inherent in the concept of EBHC, in particular in relation to translating evidence 
into action. 


METHODS. The investigation consists of a philosophical conceptual analysis, by means of epistemological issues. Specific 
kinds of knowledge inherent in the concept of EBHC will be identified, by analyzing common definitions in standard 
EBHC/EBM-literature, and explain them more detailed by using epistemological terms such as “tacit knowledge’, 
“knowledge-that” and “knowledge how” (Ryle), and of “practical knowledge” (Aristotle). Two main types of knowledge 
will be pointed out: “scientific” and “practical”, each with subgroups of more specific knowledge-types. Scientific 
knowledge is identified as reducible to the different scientific methods used in EBHC theory and practice. Practical 
knowledge will be identified through the terms “clinical reasoning”, “clinical expertise”, and “integration (of evidence, 
patient's preferences, and context)”, and clarified through epistemological terms. The concept of evidence translation 
implies certain combinations of knowledge-types. 


RESULTS. - Identification and clarification of the different types of knowledge inherent in the concept of EBHC. - 
Clarification of the role of practical knowledge in EBHC, in particular in relation to translating evidence into action. 


LIMITS. Philosophical conceptual analysis is a formal method, not directly linked to clinical practice. 


CONCLUSIONS. - There are several kinds of knowledge at play in the concept of EBHC. Of these kinds, scientific 
knowledge is the most explored. The content and implications of practical knowledge is less explored, although it is, in 
most cases, just as central. - Knowledge translation requires specific knowledge types, constituted by scientific and 
practical knowledge. - An adequate understanding of the specific knowledge types constitutive of both scientific and 
practical knowledge in EBHC should be a necessary element in EBHC education. 
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31. Assessment of reporting EBHC e-learning interventions in a Campbell systematic review 


Rohwer Anke?, Motaze Nkengafac Villyen’, Rehfuess Eva’, Young Taryn* 


1Centre for Evidence-based Health Care, Faculty of Medicine and Health Sciences, Stellenbosch University, ?Institute for Medical 
Informatics, Biometry and Epidemiology, Ludwig-Maximilians-University 


BACKGROUND. We conducted a Campbell review on e-learning of evidence-based health care (EBHC) to increase EBHC 
competencies in healthcare professionals. Data was extracted based on our logic model and included details of the 
intervention, educational context and implementation. 


AIMS. To assess reporting of EBHC e-learning interventions for studies included in our review using the guideline for 
reporting evidence-based practice educational interventions and teaching (GREET). 


METHODS. The GREET checklist comprises 17 items recommended for transparent reporting of EBHC educational 
interventions. Two authors independently assessed reporting of EBHC e-learning interventions for each of the included 
studies. We made judgements on adequate reporting for each GREET item (yes/no/unclear) and provided justifications. 
Discrepancies were resolved through discussions and consultations with a third author. We entered data into Excel and 
analysed results descriptively. 


RESULTS. Of the 24 included studies, 96% provided a brief description of the educational intervention; 29% mentioned a 
theory; 38% described the learning objectives; 92% listed the EBHC content; 58% adequately specified learning materials; 
88% described educational strategies; 25% reported incentives; 17% provided details on instructors; 71% adequately 
reported on delivery of the intervention; 46% reported learning environments; 67% described the schedule; 63% specified 
the time spent; 21% reported planned, but 0% reported unplanned changes to the intervention; 13% reported learners’ 
attendance; 4% included a process to determine whether materials and strategies were delivered as planned; and 0% 
described whether the intervention was delivered as scheduled. None of the included studies adequately reported on all 
items. 


LIMITS. We assessed reporting of interventions with GREET after our review was published and were therefore not able to 
incorporate our results into the review. 


CONCLUSIONS. Our assessment of reporting of EBHC e-learning interventions revealed that included studies did not 
follow the GREET format. Transparent comprehensive reporting of interventions is important to those considering the use 
of research on these interventions as well as for the conduct of evidence synthesis. 
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32. Evidence review for quality improvement innovations: a test of a responsive review method 


Rubenstein Lisa’, Danz Margie’, Lim Yee-Wei?, Shanman Roberta?, Motala Aneesa?, Stockdale Susan’, Shekelle 
Paul?, Hempel Susanne? 


1RAND Corporation Santa Monica, *Veterans Health Administration, Greater Los Angeles 


BACKGROUND. Given rapid changes in healthcare, effects of local environmental and organizational contexts on how 
well care is delivered, and failures to translate many effective interventions into routine practice, local quality 
improvement (QI) initiatives will be critical for advancing achievement of high value healthcare. Without objective 
evidence review, however, QI innovators may miss previously tested approaches, rely on biased information, or use 
personal preferences in designing and implementing local QI initiativesAlthough rapid review methods address 
timeliness, cost, and objectivity, these methods do not explicitly respond to QI innovator needs for seeking the best 
information available to solve a local problem. Additionally, information on e.g. tools for supporting implementation or 
important aspects of context may shape innovator assessments of feasibility. A method that makes use of high level 
evidence review capabilities while maximizing responsiveness to QI innovators is needed. 


AIMS. Develop and test an evidence review strategy specifically tailored to support local QI innovation, termed 
Responsive Innovation Evidence Review (RIER). 


METHODS. INTERVENTION: The RIER team receives a request from a QI team that identifies a quality problem, the 
proposed innovation, and the QI evaluation strategy. The RIER team then 1) does a preliminary literature scan, and 
conferences with innovators regarding their specific questions; 2) develops a parsimonious search filter to identify related 
publications with high specificity, including through specialist databases and functions (such as the Related Citations 
function in PubMed); 3) identifies any prior relevant systematic reviews; 4) looks for additional information related to 
innovation design and tools including through Google Scholar; and 5) summarizes the resulting information in short, 
highly structured, transparent, user-friendly topic briefs, with evidence tables. DEVELOPMENT: We iteratively developed 
RIER as part of a larger evidence-based quality improvement (EBQI) project aimed at transformation of Veterans Health 
Administration primary care to a patient centered team-based model. Six different Veterans Health Administration 
primary care practices within three different Southern California practice networks participated in EBQI. Frontline 
interdisciplinary EBQI teams in these practices submitted innovation proposals to system leaders for endorsement. 
Approved proposal teams could submit requests for a RIER. All EBQI innovation teams were exposed to RIERs during 
annual across-site in person learning sessions. EVALUATION: Included review of study records for scope, penetration, and 
RIER completion and an online survey of all QI innovation team leaders, whether or not they had requested a RIER, to 
assess acceptability and usefulness. 


RESULTS. PROJECT SCOPE AND PENETRATION: 60 innovator proposals were reviewed by regional leaders between 2011 
and 2012, and 15 were approved. 13 approved innovation teams requested evidence reviews during a 16 month period. 
An additional seven innovation requests from EBQI site teams came directly to the RIER team between 2012 and 2014, 
outside of the regional review process. At innovators’ requests, the evidence review team provided online general 
guidelines and tips on how to review literature. ASSESSMENT OF RIER COMPLETION AND CHARACTERISTICS: ALL RIERs 
were completed within 2 to 6 weeks by an experienced evidence review team, including a librarian. Topics included e.g. 
improving access to care, clinician burnout, nurse disease manager roles, and reducing walk-in care. RIERs, including 
evidence tables, ranged from six to 15 pages with mean of 9 pages, 1171 words, and 16 citations . Some reviews found 
little evidence of support for the proposed innovation. SURVEY ASSESSMENT OF USER ACCEPTABILITY: 17 of 28 invited 
innovators (61%) answered the survey. Over 80% of respondents had some experience with performing literature reviews; 
71% had used PubMed or other academic databases. Nearly 60% had requested or received help with literature reviews 
from the EBQI project reported here prior to the creation of the RIER team. 81% rated RIERs as very or probably useful. 
26% found the process of requesting and receiving the reviews unclear or not working well. Comments on the survey 
indicated that innovators desired more interaction with the RIER team. 


LIMITS. Participating QI teams were from a limited number of VHA practices; results may be different in other settings. 
Our evaluation survey may be biased by social desirability. 


CONCLUSIONS. These responsive evidence reviews were a key component of a multi-site evidence-based quality 
improvement intervention. The RIER approach appears promising for integrating evidence review into QI, and thus for 
enhancing the efficiency and effectiveness of local QI innovation through better access to scientific results. Future work 
on how best, and most cost-effectively, to communicate with QI teams is needed. 


Corresponding author: Rubenstein Lisa - RAND Corporation — Email: rubenstein15501@UCLA.edu 


tt] GIMBE 38 


EVIDENCE FOR HEALTH 


The ecosystem of evidence 
wy Connecting generation, synthesis and translation 


33. Training the current and future media professionals in Croatia on identifying, appraising and 
using reliable health-related information 


Sambunjak Dario, Puljak Livia? 


1Catholic University of Croatia, University of Split School of Medicine 


BACKGROUND. The majority of population in the world today is regularly exposed to media content, which also includes 
the information about health and disease. Especially when faced with health problems, people tend to be receptive to 
media messages related to their health condition. However, it is not always easy for consumers to discern a reliable 
health-related content from sensationalist half-truths or biased information presented in various media channels. Even 
when media are well-intended, the journalist working for them may not be fully aware of the impact of their work on 
public opinion and behaviour, nor equipped with necessary knowledge and skills for appraising and processing the 
health-related content. Therefore it is important to raise this awareness and build the necessary knowledge and skills 
among the media professionals, preferably in the earliest stages of their careers. 


AIMS. The aim of this project was to train journalists and journalism/communication science students in four major 
university cities in Croatia to identify, appraise and use reliable health-related evidence. The purpose was to contribute to 
creating a cohort of current and future media professionals that will be able to responsibly and accurately present the 
health-related information to consumers. 


METHODS. This project was supported by Ministry of Science and Education of the Republic of Croatia, through a 
programme aimed at popularization of science among general public. Training materials were created for a workshop- 
type activity with students. Two-hour workshops were organized at five Croatian universities in Zagreb, Split, Osijek and 
Dubrovnik. The target audience were journalists and students of media-related studies. 


RESULTS. Three workshops were delivered in March 2017, first with students of journalism at the University of Zagreb, 
second with the students of communication sciences at the Catholic University of Croatia in Zagreb, and third with the 
students of cultural sciences at the University Josip Juraj Strossmayer in Osijek. Two more workshop are to be conducted 
in April and May at the University of Split School of Medicine and at the University of Dubrovnik. Altogether, about two 
hundred of participants are expected to attend. All the workshops were delivered by the same expert in evidence-based 
medicine and science communication. Workshops consisted of an hour of presentation about the principles of evidence- 
based medicine and communicating reliable health-related information, followed by an hour of group work whereby the 
participants discussed the examples of good and poor practice, based on the real cases of media reports. At the end of 
the workshop, the participants received a Croatian-language copy of ‘I've got nothing to lose by trying it’ booklet 
produced by Sense about Science charity, which was translated into Croatian and printed by Cochrane Croatia. 


LIMITS. Although the target audience of this project included all current and future media professionals, the majority of 
the workshop attendants were students in media-related study programmes. This is in part due to the fact that in Croatia 
there is only a very small number of active journalists exclusively covering health- and science-related topics. This means 
that possible impact of this kind of training may not be expected immediately, but only after the student complete their 
studies and start their careers in media industry. Also, the coverage of relevant students in Croatia was not complete, as 
there are several universities that were not covered by this project, due to limited funding. 


CONCLUSIONS. Training of current and future media professionals in identifying, appraising, and presenting reliable 
health-related information is not part of curricula of media-related study programmes at Croatian universities. Specific 
and targeted workshops such as the ones conducted in this project can be used to overcome this gap. Ultimately, this 
should improve the reliability and quality of media reports based on which many consumers make their heal-related 
decisions. 
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34. A new instrument to assess the trustworthiness of effect modifiers 
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McMaster University, ?Cochrane China, *Basel University Hospital 


BACKGROUND. Debates regarding the credibility of effect modifiers are often contentious. Although it is desirable to 
identify effect modifiers that explain heterogeneity of treatment effects, subgroup analyses may lead to spurious 
inferences of subgroup effects in randomized trials and meta-analysis. Authorities have, in response, suggested varying 
criteria to assess the credibility of effect modifiers. A formal, consensus-based instrument remains unavailable. 


AIMS. To develop an instrument to assess the credibility of putative effect modifiers in randomized trials and meta- 
analyses 


METHODS. We will follow a rigorous instrument development process, which will involve expert panels and users. First, 
we will perform a qualitative systematic survey of the methodological literature discussing credibility of effect modifiers. 
We systematically searched Medline, Embase and Textbooks and identified 409 potentially relevant full texts. These 
reports will serve as the basis for identifying experts and generating candidate items for the new instrument. We are 
currently abstracting reported credibility criteria (e.g. pre-specification, test of interaction, small number of subgroup 
analyses) and the methodological context (e.g. trials or meta-analysis, purpose of subgroup analysis, concept of effect 
modification). We will randomly choose 20 experts who will form two panels. Panel 1 will be involved in the instrument 
development and panel 2 in the testing phase. In addition, we will involve two groups of 20 users who will apply the draft 
instrument to a sample of subgroup analyses using formal user testing methods. 


RESULTS. At the time of the conference, we will present the concept, the item selection process, and the draft instrument. 
The new instrument will have immediate impact on the analysis, interpretation, and reporting of effect modifiers in 


individual trials and meta-analysis. 
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35. Developing EBP competencies in nursing education 
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Rotterdam University of Applied Sciences 


BACKGROUND. There has been an increasing emphasis on Evidence Based Practice (EBP) in Applied Sciences in 
healthcare education, such as nursing. However, although several studies and programs showed positive effects of EBP 
on healthcare quality and patient outcomes, it is still not standard practice in daily nursing care. To highlight the 
importance of EBP and make certain that nurses actually adopt EBP competencies in real-world healthcare settings, a 
different approach, starting from their education on, is needed. The level of EBP competencies in nursing students is 
unknown as well as the effect of current curricula on their motivation to use EBP competencies. Both EBP attitude and 
competencies have been examined previously in students of allied health care studies; speech-language therapy, 
occupational therapy and physical therapy. 


AIMS. We aim to assess the effect of the current curriculum in third year nursing-students on the motivation to use EBP 
and their EBP competencies and if a relationship can be found between EBP attitude and competencies. Nursing students’ 
motivation and competencies will also be compared with those of other healthcare students. Finally, we aim to identify 
any possible gaps and recognize how to upgrade the current curriculum towards a better and more integrated education 
of EBP. 


METHODS. At the beginning of the first semester of their third year, 98 nursing students of Rotterdam University of 
Applied Sciences filled out a questionnaire. 91 students agreed to use their data in our analysis. Data regarding EBP 
knowledge and skills was acquired by using the Dutch Modified Fresno test (DMF) (Spek et al., 2012). Students were, for 
example, asked to formulate a correct research question or to define reliability. To make the DMF test more suitable for 
nursing students, case vignettes were adapted to their professional field. In addition, the motivational beliefs of EBP were 
measured by the Dutch Motivational Questionnaire (DMQ) (Spek et al., 2013) consisting of two subscales: task-value and 
self-efficacy. Answers were given using a seven-point Likert scale (-3=strongly disagree, O=neutral, 3=strongly agree). 
SPSS Statistics 23 is used to analyze the relation between attitude towards EBP and EBP competencies and to identify 
differences with the allied health care students. 


RESULTS. Preliminary results show that most nursing students (94,3%) scored low on task-value (M=1,83 SD=0,73 N=87). 
Students seem to be positive about their self-efficacy in EBP (M=2,54 SD=1,16 N=87) (lower scores on the self-efficacy 
subscale represent higher self-efficacy). In the total group, no relation was found between motivational beliefs of EBP 
(DMQ) and EBP competencies (DMF). In students with a higher self-efficacy score, a positive correlation (r (38) = .33, p< 
.05) was found between self-efficacy (M=1,79 SD=0,62 N=38) and knowledge of EBP. (M=12,47 SD=7,16 N=38). Students 
with a higher self-efficacy seem to perform better on these questions. Knowledge regarding effect size-measures — 
absolute risk reduction (ARR), relative risk reduction (RRR), and number needed to treat (NNT) — seem to be insufficient in 
nursing students. All 91 students except for one failed to answer this correctly. Further results of this study will be 
available and discussed at the EBHC conference. 


LIMITS. The data presented in this study is a first baseline measurement. This study is part of a longitudinal study covering 
all allied health care and nursing curricula. A second limitation is related to the number of students (N=91) that 
participated in this baseline measurement. Almost a third of third year students could not participate due to internships. 


CONCLUSIONS. In order to review EBP in the curriculum, measuring EBP knowledge and skills as well as motivational 
beliefs is important. As seen (so far) in this study, nursing students scored low on task-value of EBP. Knowledge on some 
EBP-topics was insufficient. The DMF test, combined with the questionnaire on motivational beliefs are useful instruments 
to evaluate the attitude and competencies in EBP. Important to take into consideration is that not all EBP competencies 
are necessary for nurses to be considered competent evidence-based practitioners. Changes in the curriculum regarding 
EBP need to be aligned with real-world healthcare settings. 
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36. Developing clinical practice guidelines in Uzbekistan: collaborative efforts between the 
Centers for Disease Control and Prevention and the Ministry of Health 


Sharapov Makhmudkhan’, Schmid George’, Cope James”, Karamatova Shakhida?, Jabriev Jamshid? 
ICDC office in Uzbekistan, ?CDC (USA, ATL), CDC (USA, ATL), CDC office in Uzbekistan, ?Project "Health 3"of MOH of Uzbekistan 


BACKGROUND. The Soviet-era approach to providing guidance for the management of health-related problems was 
dictatorial and nonevidence-based. While providing uniformity, the results were marked by ineffectiveness, inefficiency, 
and inflexibility. Separation of the Russian-speaking Soviet countries from the influences of Western- and English-based 
thinking led to the rigid imposition of a panoply of disparate and noncomprehensive documents called prikaz’ (decree) 
for the diagnosis and management of individual health or public health conditions. Creativity and flexibility were stifled 
because of a culture of blame and punishment if prikaz’ were not followed rather than reward for the identification of 
gaps in the breadth of coverage of prikaz’ or their incorrectness. Subsequent to the fall of the Soviet Union, the rigid 
structure of the prikaz was followed in many ex-Soviet countries but was then influenced by pharmaceutical companies, 
various medical institutes and groups of professors with particular interests, and continuing isolation from Western- and 
English-based thinking based on evidence, effectiveness, and efficiency. In 2013, the American Centers for Disease 
Control and Prevention (CDC) and the Uzbekistan Ministry of Health (UMOH), with support from the World Bank that 
funded a small health team (“Health 3”), began a long-term project to introduce the concept of Clinical Practice 
Guidelines (CPGs) based on evidence and to revise the then-existent approach of Uzbekistan to providing guidance for 
the identification and management of individual- and public health-related problems. 


AIMS. To assess the structure, content, and design of existing guidance of the UMOH and propose a new approach to the 
development of CPGs, prepared on the basis of modern international standards. 


METHODS. No 


RESULTS. Recognizing a general problem, the UMOH tried more than 10 years ago to improve the lack of guidance based 
on evidence. Initially, the United States Agency for International Development helped create Centers of EBM in two 
Medical Institutes which developed four CPGs with the technical assistance of UNICEF, in 2005 and 2011. Although one 
Center survived following withdrawal of funding, it was not active, little further work ensued, and EBM approaches to 
guidance lay dormant. CDC and UMOH began its work in 2013 by cataloguing existing guidance and found innumerable 
products with the varying designations of “Standards of diagnosis and treatment,” “Clinical protocols,” “Practice 
guidelines,” “Methodological letters for treatment,” “Guidelines for best practice,” etc. Scientists and leading specialists 
from Medical Institutes, Research Medical Institutes and Republican Medical Centers were authors (usually 2-4 people) of 
these products, each with different formats, and none attractive nor “physician-friendly.” Diagrams and tables were 
seldom used with, instead, multiple pages of text. Recommendations were not consistent with the principles of EBM, 
frequently reflecting opinions of experts or authoritative scientists to the detriments of facts, and legalizing doubtful 
practices. There was no information about the process of development, nor methods of distribution and implementation. 
Indeed, most documents could not be put into practice because of the absence of demand from practitioners and their 
isolation from the real situation of medical practice. Cost-effectiveness was never considered nor were preferences of 
patients. Approaches were often outdated and there was no update mechanism. Having performed a situation analysis, 
CDC and UMOH devised a plan to institute a standardized approach to the development of CPGs and a standardized, 
appealing format featuring diagrams and algorithms. During and following three CDC workshops, participants evaluated 
12 local CPGs using the AGREE II instrument. All CPGs received low scores. We are now establishing three 
multidisciplinary working groups of (8-10 experts in each groups) to devise a CPG for each of three diseases which are 
most important to public health. A future fourth training will review them, reinforce principles, and revise them before 
beginning further CPGs. 


LIMITS. While the UMOH has been quite supportive of our collaborative effort, working within the framework of a 
decades-old culture and approach to medical guidance has proved challenging when attempting to change both. The 
lack of English-speaking colleagues in Uzbekistan and lack of internet accessibility or use continues to hamper our 
efforts to introduce change. 


CONCLUSIONS. We believe strongly in the need for CPGs in Uzbekistan to change to those based on evidence and with a 
uniform, appealing format, with effectiveness, cost-effectiveness, and patient-consideration being guiding principles. We 
believe the difficulties we have encountered are shared by many other ex-Soviet countries, including Russia, and that our 
experience in introducing such change can be shared with them. 
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37. EBP profiles among bachelor students in health care: is there a difference between health 
disciplines? 


Snibsger Anne Kristin’, Graverholt Birgitte?, Nortvedt Monica Wammen?, Riise Trond?, Espehaug Birgitte? 


1Western Norway University of Applied Sciences, *University of Bergen 


BACKGROUND. New demands in health care impose changes in health care education and training, and there is a 
progressive focus on EBP teaching and learning within health care educations. Despite the recognition of integrating EBP 
in educational programs, there is limited research about bachelor students’ EBP knowledge, attitudes and behavior 
across health disciplines. 


AIMS. The aim of this study is 1) to examine EBP profiles among bachelor students in health care and compare profiles 
between health disciplines, and 2) to explore the relationship between EBP profiles and bachelor students’ assessment of 
EBP teaching and expectations from teachers. 


METHODS. A survey was conducted among bachelor students in health care during spring 2015. Final year students in 
nursing, occupational therapy, physiotherapy and radiography from four educational institutions in Norway were invited 
to answer the ‘Evidence-Based Practice Profile - Norwegian version’ (EBP2-N). The questionnaire assesses the five steps of 
evidence-based practice and consists of items related to the five domains of Relevance (value range 14-70; higher scores 
indicate a higher degree), Terminology (17-85), Confidence (11-55), Practice (9-45) and Sympathy (7-35). In addition, 
information was rendered on satisfaction with EBP teaching (value range 5-20; higher scores indicate a higher degree) 
and assumed expectations from teachers of EBP performance (5-10). Linear regression analyses, simple and with 
adjustment for educational institution, gender, work and age, were performed to investigate mean differences in EBP2-N 
domains between health disciplines. We calculated Cohen’s d to illustrate the magnitude of the largest difference for 
each domain. Spearman’s rho (rs) was used to assess the monotonic relationship between the EBP2-N domains and 
students’ satisfaction with EBP teaching and assumed expectations from teachers of EBP performance. 


RESULTS. Of the 775 students who met to the teaching sessions, 713 (92 %) answered the questionnaire. Overall, mean 
score for Relevance was 59.5 (Cl 95% = 59.0-60.0), for Terminology 47.0 (46.1-47.8), for Confidence 34.8 (34.3-35.3), for 
Practice 23.8 (23.4-24.2) and for Sympathy 21.8 (21.5-22.1). Statistically significant differences (p < 0.05) between health 
disciplines were observed for all domains. Students in radiography reported mean scores below the overall average on 
all EBP2 domains. The largest mean difference was found for Relevance with highest score for occupational therapy and 
lowest for radiography, with an estimated Cohen’s d of 1.13. Positive correlations were observed between Relevance and 
satisfaction with EBP teaching (rs= 0.31), and between Relevance and expectations from teachers of EBP performance 
(rs= 0.36). The strongest correlation was observed between Confidence and satisfaction with EBP teaching (rs = 0.46). 


LIMITS. We used a self-reported scale. The questionnaire was validated, but the results of Sympathy and Practice should 
be interpreted with caution, as structural validity was not confirmed for these domains. 


CONCLUSIONS. Bachelor students in health care found EBP relevant, but revealed mediocre confidence, understanding of 
common EBP terminology, and use of EBP in clinical situations. We also observed differences in EBP profiles between 
health disciplines. Furthermore, the findings suggested an association between high scores on Relevance, Confidence and 
satisfaction with EBP teaching, and between Relevance and expectations from teachers of EBP performance. 
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38. The challenges of evidencing education and support interventions 


Steven Alison’, Larkin Val’, Redfern Nancy’, Wilson Gemma}, Stewart Jane? 


1Northumbria University, 7?Newcastle Hospitals NHS FoundationTrust, Newcastle University 


BACKGROUND. Developing evidence can be difficult, especially in relation to complex social interventions. Activities such 
as mentoring which relate to education, staff support and development fall into this category. Here outputs are often co- 
constructed and abstracted, and causal relationships difficult to prove in terms of traditional evidence hierarchies 
requiring demonstration of measureable effect. Although such educational interventions require rigorous examination 
questions arise: how do we build a robust evidence base in a sector with limited funding and where most inquiry is 
context dependent and small-scale? How do we explore and review emerging evidence that is ‘weak’ by RCT standards 
and in some instance not research based? Do we ignore these until ‘research’ is undertaken? Or utilise, in a robust 
manner, emerging hunches and multiple anecdotes which, over time, illuminate the complex social intervention and 
suggest ‘effect’? How then could researchers systematically gather, evaluate and condense this nebulous evidence? 
Members of our team have been involved in mentoring activities and small-scale research into mentoring for many years. 
Activities include developing mentoring programmes and schemes, training and educating mentors, undertaking 
evaluation research into schemes, programmes and the experiences of those involved. During this time, although not 
seeking it, we have consistently noted in ours and other’s research data and anecdotal evidence from experience possible 
links between involvement in mentoring activities and doctors health and wellbeing. Many emerging links are positive 
but some are negative. We became increasingly cognisant of this unanticipated, unsought ‘effect’. Thus, we applied for 
and were successful in gaining funding for a 3 year project from the British Medical Association to explore these links 
through a multi stage study. 


AIMS. This paper will describe and discuss issues related to developing evidence and evidence synthesis in a field of study 
with little traditional research and where the phenomenon is complex, socially bound and interpreted in multiple ways. 
We will use our recent experience of undertaking a systematic narrative review into the links between doctors’ health 
and wellbeing and involvement in mentoring activities to illustrate the issues. This will include the challenges of 
publishing work not fitting traditional models. 


METHODS. Search terms were developed using the PICO framework1 . We systematically gathered literature via searches 
across 12 databases over 10 years (January 2006 - January 2016). Truncation and key terms were searched for in the title, 
abstract and keywords. Only English language was included. Items were included if they involved doctors’ engagement in 
mentoring activities and, either health or well-being, or the benefits, barriers or impact of mentoring. Literature reviews 
were excluded as they did not provide original evidence and care was taken not to double count sources. Initial searches 
found 4669 papers, after exclusions and full-text analysis of 37 papers, 13 were accepted for review. Paper quality was 
examined using the Critical Appraisal Skills Programme tool. Theory-driven thematic analysis was undertaken. The BITC 
Workwell Model3 acted as a heuristic device to synthesise findings into 4 groupings; better relationships, better physical 
and psychological health, better specialist support and better work. 


RESULTS. Papers reviewed suggest mentoring contributes to doctors’ health and well-being by enhancing relationships, 
physical and psychological health, specialist support, and may lead to better work. Findings suggest mentoring influences 
professional and personal relationships due to increased collegiality, networking opportunities, and development of 
transferrable communication skills. Relationships with key individuals and working in supportive teams, may impact on 
stress by influencing levels of social support and role clarity. 


LIMITS. We acknowledge this is a small, tentative study in an uncharted area and may be viewed by some as lacking in 
value. 


CONCLUSIONS. The systematic narrative review is not a traditional systematic review of effect, it is however we believe 
rigorous and systematic, and a small step in trying to ascertain if links between mentoring and doctors’ health and 
wellbeing exist. It is an additional tool for building evidence in a field akin to medical education research which does not 
lend itself to ‘traditional approaches’. When seeking publication barriers were evident, with some journals reticent to 
consider publication of innovative techniques for exploring ‘evidence’. Not conforming to the norms of the scientific 
method is likely to be judged at best as different and at worst lacking quality and rigour. Whilst practitioners may be 
ready for new ways of thinking about evidence, publishers may be more reserved. 
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published by USA non-governative organizations 
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1University of Southern California, Our Lady of the Lake Hearing and Balance Center, Washington University Program in Physical 
Therapy, “Jewish Rehabilitation Hospital, *Rockhurst University, Symmetry Alliance, LLC, 7Research Institute of Chicago, "James A Haley 
Veteran's Hospital 

BACKGROUND. Over 90% of clinical practice guidelines (CPGs) published in the United States are published by non-profit, 
non-governmental organizations. Over the past decade many such organizations have built their capacity to publish such 
guidelines, often through volunteer development groups. However, few of these organizations have developed the 
capacity to effectively disseminate their CPGs much beyond publication in a peer-reviewed journal. Volunteer CPG 
developers for these organizations may not have expertise in dissemination and implementation and are often ready to 
move on to new projects after an intensive 3-5 years of CPG development. Models are needed to facilitate broad-scale 
dissemination and stakeholder-informed implementation of non-profit, non-governmental organizations’ CPGs. 


AIMS. 1) To report a process established by a task force of the Academy of Neurologic Physical Therapy of the American 
Physical Therapy Association to disseminate and promote implementation of a CPG. 2) Propose a model for similar 
organizations to follow after publication of CPGs. 


METHODS. A call for volunteers to join a Dissemination Task force was sent to organization members two months prior to 
publication of the target CPG. Within one month of the CPG’s publication, a seven-member volunteer task force was 
selected and charged to disseminate the CPG and promote its implementation among clinicians in the United States. 
None of the members were authors of the original CPG, all were active practitioners in the CPG focus area (vestibular 
disorders), and one had an academic focus in implementation science. Three members, new to implementation science, 
attended a 2-day knowledge translation workshop to broaden the groups’ expertise. The task force was convened for 3 
years and was provided with a budget between $10,000 and $15,000 (USD). 


RESULTS. The task force met monthly via teleconference convened by the chair. A three-phase plan was developed and 
initiated: Phase 1) Rapid, wide-spread dissemination to promote awareness of the CPG among primary intended users (specialty 
practice physical therapists and physicians); Phase 2) A multi-site implementation study at the task force members’ institutions 
(ranging from small private to large academic and government healthcare institutions); and Phase 3) Sustained, wide-spread 
dissemination of products developed and lessons learned from the implementation phase for institutions across the country. 
Phase 1 resulted in development of at-a-glance CPG summaries for key stakeholders (i.e. physical therapists, physicians, and 
patients) and a decision aide for physical therapists. The documents were distributed in mass at targeted professional 
conferences, online, and via social media. Phase 2 resulted in a five-site implementation study based on the Consolidated 
Framework for Implementation Research and the Knowledge to Action Cycle. Results presented will include how the 
Consolidated Framework and Knowledge to Action Cycle influenced the process of selecting and tailoring the implementation 
intervention and ongoing results of monitored behavior change. The impact of emerging Phase 2 results on Phase 3 plans will 
also be presented. Phase 3 is expected to include dissemination of common barriers with suggestions for solutions, audit and 
procedural monitoring checklists, and how-to manuals and videos. These will be hosted on the CPG author organization's 
webpage, built into the organization’s sponsored continuing education courses, and presented and distributed at professional 
conferences and through social media outlets. 


LIMITS. The task force is one-year into its three year process. However, it has completed development of a novel model 
that other non-profit, non-governmental organizations can learn from. Presentation of the model and its results to date 
will facilitate valuable discussion about new methods for evidence translation. 


CONCLUSIONS. Non-profit, non-governmental organizations publish the vast majority of CPGs in the United States (and 
the world). Our model provides a systematic process by which these organizations may build their capacity to 
disseminate and promote implementation of their CPGs. A task force, separate from the CPG development team, 
composed of expert clinicians and individuals with expertise in implementation science is recommended. A 3-phase, 
sequential process facilitates integration of broad dissemination and locally adapted, stakeholder-informed 
implementation efforts as follows: 1) broad dissemination for awareness; 2) task force member-initiated implementation 
efforts at their local institutions; and 3) sustained, wide-spread dissemination of lessons and products from the 
implementation efforts in Phase 2 that can be adapted for local use. Early results from this process are promising and 
further attention is warranted. 


Corresponding author: Tilson Julie — University of Southern California — Email: tilson@usc.edu 


&# GIMBE 4s 


EVIDENCE FOR HEALTH 


The ecosystem of evidence 
wy Connecting generation, synthesis and translation 


40. The co-creation of knowledge for a patient-centered integrated healthcare system 


Tsasis Peter 
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BACKGROUND. An aging population and success in treating acute events have increased the number of individuals with 
complex needs who require ongoing care from multiple health care providers. These changes necessitate well- 
coordinated, high-quality care across the health system to reduce duplication of services/waste, medical errors, and 
improve the delivery of care in order to ensure a sustainable healthcare system. Studies of inter-professional and 
interdisciplinary teamwork in healthcare indicate that collaboration is a contested process in which healthcare 
professionals from various disciplines have to learn to work together. However, the process by which this type of learning 
is enacted remains poorly understood in the delivery of integrated care to patients with complex needs where knowledge 
needs to be co-created with the patient in a patient centered approach. 


AIMS. This research examines the healthcare professional behavioral underpinnings of how learning and knowledge 
exchange processes unfold within and across inter-professional and inter-organizational teams to impact quality of 
healthcare by learning from the patient about the patient, their preferences, values, fears, wants and satisfaction as they 
relate to healthcare. 


METHODS. In depth semi-structural interviews were conducted with Community Care Coordinators embedded in multiple 
interdisciplinary teams of healthcare professionals providing integrated care to patients with complex needs. Interviews 
were complemented by observations of group meetings and listening-in on patients’ case study group discussions. 


RESULTS. Findings reveal that health system sustainability is challenged by system complexity, weak ties and poor 
alignment among health professionals and organizations, a lack of funding incentives to support collaborative work, and 
a bureaucratic environment. Policies and management practices are needed that promote system awareness, 
relationship-building and information-sharing, and that recognize delivering healthcare as an evolving learning process 
rather than a series of programmatic steps. The data highlighted that much of what makes sustainable integrated 
healthcare delivery successful or challenging is in the professional backgrounds, personalities, attitudes, preferences and 
expectations of individual providers and how each healthcare provider defines and enacts the concept of collaboration, 
patient engagement, and communication in the context of care provisions. Interdependence among healthcare 
professionals and collaboration among team members generally increased over time as team members recognized the 
value of others’ knowledge and expertise and became accustomed to drawing on that knowledge in care provisions and 
on patient experience. Learning occurred through active participation as a collective social process rather than an 
individual process without points of contact, through four inter-related behaviors: interaction, feedback, reflection, and 
self-directed learning. 


LIMITS. Limitations stem from local biases that are introduced within the context of the study in the province of Ontario, 
Canada. 


CONCLUSIONS. This research makes a number of conceptual contributions that highlight the limits of both the “best 
practice approach” and the search for a universal solution to integrated care, independent of patient experience, context 
and local contingencies of learning. The results of the study can be used to identify ways to better support healthcare 
professionals in collaborating for integrated care delivery and a sustainable healthcare system that is patient centered, 
as well as to create work environments that facilitate learning and knowledge exchange across organizational and 
professional boundaries within a patient-focused care model. By bringing together knowledge and expertise from various 
levels and sectors of healthcare, this research takes a vital step towards fulfilling important gaps in patient-clinician 
partnership formation, knowledge exchange, and coordinated actions toward increasing value, by putting the patient at 
the center of care and in providing evidence based research in healthcare delivery and communication. 
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BACKGROUND. In the UK and internationally there is widespread acceptance of the value of economic evaluations to 
inform decisions about health care interventions. The general methods of economic evaluation in health care 
interventions are now well established. By contrast, approaches to social care economic evaluation are significantly less 
well developed. In 2013, the National Institute for Health and Care Excellence (NICE) was conferred with the 
responsibility for developing guidelines and quality standards in social care. NICE commissioned this scoping review to 
support developing a long-term strategy for how to consider social care economics in guidelines. 


AIMS. The project aimed to inform NICE on the methods available for use in undertaking economic evaluation of social 
care interventions, the methods in development, the methods challenges faced and the methods gaps. 


METHODS. A systematic review of the published literature and a survey of experts were undertaken to identify key 
methods used to undertake recent economic evaluations of social care interventions. Each study was assessed in terms of 
the key requirements for economic evaluation. Data were extracted on: the perspective of the analysis, the interventions 
compared, the evidence used on costs and effects, opportunity cost, uncertainty, and equity. Expert advisors commented 
on the findings of the review and this informed the results that were drawn from the studies. Recommendations were 
made to improve the conduct and reporting of studies, and areas of further research were identified. 


RESULTS. Thirty social care economic evaluations were identified for review. Findings were reported on key requirements 
for economic evaluation comprising: the perspective of relevance to the decision maker, an evaluation comparing all 
relevant alternative interventions, use of all available evidence on costs and effects of relevance to the decision, analysis 
of whether the benefits of an intervention were greater than the forgone benefits of displaced interventions, assessment 
of the uncertainty associated with the decision, and exploration of the equity implications of the decision. 


LIMITS. Given resource constraints the empirical review and scope was time-limited. The review was limited to peer- 
reviewed publications from 2010 onwards and therefore cannot be considered comprehensive. 


CONCLUSIONS. A more developed evidence base is required in order to undertake economic evaluation of social care 
interventions. This should include undertaking primary studies where there is not sufficient evidence. Studies based on 
decision models and secondary evidence should be used where sufficient evidence is available, that is in terms of the 
value of information: does the value of the evidence expected to be generated from another primary study justify its 
costs. Methods guidance for the economic evaluation of social care interventions needs to reflect what is feasible given 
the available evidence and what is appropriate for social care. Further information is required on the objectives and the 
constraints of the decision maker(s) and how this links to outcomes, the resource use and the costs that are included in 
economic evaluations of these interventions. Development of a cost-effectiveness threshold for use in social care based 
on an empirical estimate of opportunity costs is required and this is a key priority. Further guidance on methods for the 
economic evaluation of social care will enhance comparison across evaluations. Further research is required to quantify 
and best represent cross sectoral resource use costs and outcomes to usefully inform the decision-making process, 
including consideration of users, carers, Commissioners and equity considerations. 
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42. Adherence to medicines in patients suffering from chronic conditions: beliefs about medicines 
and locus of control 
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1University of Malta, 7Mater Dei Hospital 


BACKGROUND. Prevalence of non-adherence to medicines is reported amongst many communities and is considered to 
be an important determinant of negative health outcomes. Behavioural models highlight the importance of health beliefs 
in the decision-making process of patients about their medicines. 


AIMS. To determine the relationship between adherence to medicines in patients with chronic conditions and beliefs 
about medicines and health locus of control. 


METHODS. The first 300 sequential adult patients having confirmed diagnosis of asthma, cardiovascular conditions or 
diabetes and attending the respective out-patients’ clinics were recruited to participate in a cross-sectional study. 
Patients completed a self-administered questionnaire which sought to determine adherence to medicines using the ‘Tool 
for Adherence Behaviour Screening’ (TABS), beliefs about medicines using the ‘Belief about Medicines Questionnaire’ 
(BMQ) Specific and General, and health locus of control using ‘The Multidimensional Health Locus of Control Scale’ Form 
C (MHLOC). The 8-item TABS has two subscales (adherence/non-adherence), each comprising four items answered ona 5- 
point Likert-scale. Non-adherence was considered to be a score of less than 19 for the ‘adherence’ subscale and more 
than 8 for the ‘non-adherence’ subscale. The total score for ‘non-adherence’ was subtracted from that of ‘adherence’; a 
TABS differential of 15 or more was considered as good adherence and less than 15 as suboptimal adherence in this 
study. A logistic regression was performed to ascertain the effects of each factor on the likelihood that patients are 
adherent to medicines. Difference between chronic conditions was determined by one-way ANOVA. P-values = 0.05 were 
considered significant. Ethics approval was obtained. 


RESULTS. A total of 300 patients were recruited (58% male; mean + standard deviation age: 61+15 years): 100 asthma, 100 
diabetes and 100 cardiovascular. The majority of respondents (82%) had suboptimal adherence to medicines; mean + SD 
adherence score on the ‘adherence’ subscale was 18.1+2.2 (41% non-adherent) and mean adherence score on the ‘non- 
adherence’ subscale was 8.2+3.2 (44% non-adherent). Logistic regression model for MHLOC and BMQ covariates in 
relation to overall adherence was statistically significant, ?2(11) = 57.468, p = .000. Higher ‘specific concern’ beliefs were 
associated with an increased likelihood of non-adherence (p = .000). The mean + SD necessity—concern differential was 
4.54+6.82. For 24% the necessity score was lower than concerns score and for 9% it was equal. Logistic regression model 
for MHLOC and BMQ covariates in relation to ‘adherence’ sub-scale was statistically significant, ?2(9) = 65.519, p = .000. 
Higher ‘general harm’ (p = .000) and ‘specific concern’ beliefs (p = .002) were associated with an increased likelihood of 
non-adherence (p < 0.05). Overall, patients had strong doctor beliefs 15.27+2.70; however, lower ‘doctor beliefs’ were 
associated with an increased likelihood of non-adherence (p = .012) on the ‘adherence’ sub-scale. Logistic regression 
model for MHLOC and BMQ covariates in relation to ‘non-adherence’ sub-scale was statistically significant, ?2(9) = 96.671, 
p = .000. Higher ‘general overuse’ (p = .003), ‘specific concern’ (p = .000) and ‘other people’ (p = .000) beliefs and lower 
‘internal beliefs’ (p = .031) were associated with an increased likelihood of non-adherence. There was no statistically 
significant difference in adherence between the three chronic conditions. However, the ‘general harm’ belief was 
statistically significantly lower for the diabetes group (p = .042) when compared to cardiovascular patients. Also, the 
‘specific concern’ belief was statistically significantly higher for the asthma group when compared to cardiovascular 
patients (p = .015) and when compared to the diabetes group (p = .050). ‘Internal’ belief was statistically significantly 
lower for the asthma group when compared to cardiovascular patients (p = .000) and diabetes patients (p = .001). The 
male gender (p = .003), younger patients (p = .005) and the lower level of education (p = .021) were associated with an 
increased likelihood of non-adherence. 


LIMITS. While participants seem to be similar to other populations, caution should be exercised in extrapolating the 
results to other communities in view of the differences in healthcare systems, practices and cultures. 


CONCLUSIONS. Higher concern about medicines, greater perceived medicine harm and overprescribing of medicines by 
doctors translated to lower medicines adherence. Beliefs vary also in between different chronic conditions. Interventions 
targeting adherence need to recognise patients’ beliefs about their conditions and medicines to influence adherence behaviour. 


Corresponding author: West Lorna Marie — University of Malta — Email: lorna.m.west@gov.mt 
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43. Review strategies to inform research prioritization of biomarkers: AKI-Diagnostics case study 


Wright Judy}, Mitchell Liz, Smith Alison?, Hall Peter?, Messenger Michael?, Wickramaskera Nyantara?, Vinall- 
Collier Karen?, Lewington Andrew3, Calder Nicola 


1University of Leeds, University of Leeds, University of Leeds, University of Edinburgh, University of Leeds, University of Leeds, University 
of Leeds, *Leeds Teaching Hospitals NHS Trust, University of Leeds 


BACKGROUND. Acute kidney injury (AKI) occurs in many critically ill patients and leads to poor clinical outcomes and 
high mortality and cost to the NHS. Numerous biomarkers for the early diagnosis and monitoring of AKI have recently 
been developed. Evidence on their clinical validity and utility is variable, and there is a time-limited opportunity to 
propose an efficient future research strategy for AKI diagnostics in the UK, to inform optimal test-reimbursement 
decisions. 


AIMS. To identify, prioritise and evaluate AKI in-vitro diagnostic tests for use in the critical care setting using a two-stage 
systematic review method. 


METHODS. Stage 1 consisted of a broad scoping search of world literature to identify candidate biomarkers for 
evaluation. The results were used to produce a ranked shortlist of priority biomarkers according to criteria agreed via 
expert consensus: volume and currency of evidence, number of samples studied, and biological plausibility. Stage 2 
consisted of a systematic review to identify evidence on the analytical and clinical validity, and clinical utility of the 
priority biomarkers. 


RESULTS. Over 150 in-development tests were identified in stage 1. The scoping search identified 4,804 references. After 
screening by titles/abstract, 487 potentially relevant papers remained, relating to 152 individual biomarkers. Those 
already used in standard care (11; including serum creatinine) or with incomplete data related to the dimensions outlined 
above (19) were excluded. Ten priority biomarkers/tests were shortlisted: BNP, Cystatin C, IL-6, IL-18, KIM-1, L-FABP, NAG, 
Nephrocheck®, NGAL and TNF-a. Stage 2 systematic review searches identified 5,045 citations for the 10 biomarkers. 
Three were selected for screening and data extraction: Nephrocheck® (which has received significant marketing and is 
the only FDA licensed test), Cystatin C and NGAL (both with the greatest volume of evidence). Data was extracted for 207 
eligible papers and 61 papers were included in the synthesis: 5 Nephrocheck®, 17 Cystatin C, 39 NGAL. Methodological 
challenges were identified in searching for diagnostic studies which were found to be poorly reported and indexed in 
databases. Extra resources were required for screening and data extraction due to the large numbers of records, poor 
reporting, complex medical and diagnostic terminology, studies using multiple tests in conjunction and the complexity of 
including both clinical and analytical validity in the synthesis. 


LIMITS. Limits 1) In the absence of published guidance, the test shortlisting criteria were developed by expert 
consultation, and may not capture promising in-development tests due to the pragmatic focus on objective criteria (e.g. 
volume of evidence). 2) The volume of evidence was substantially greater than originally indicated by pre-study scoping 
searches, largely due to the decision to broaden the final scope to include tests developed outside the critical care 
setting. Together with the number of candidate tests identified (including multiple tests used in conjunction) and the 
complexity of data extraction involved, this resulted in extended study timelines. 3) Poor reporting, especially of 
analytical factors, makes comprehensive synthesis of test analytical and clinical validity difficult. 


CONCLUSIONS. As the number of biomarkers entering the healthcare market continues to rapidly expand, the role of 
reviews to inform future research priorities is becoming increasingly important. The two-stage search process outlined 
here represents a novel approach in this area; however, it is clear that further work is required to establish efficient and 
optimal search strategies, data extraction for clinical and analytical validity in studies of multiple tests, and shortlisting 
criteria for such reviews. 


Corresponding author: Wright Judy — University of Leeds — Email: j.m.wright@leeds.ac.uk 
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44. A bayesian model for evidence-based shared decision-making 


Achilleos Haris 
Imperial College Healthcare NHS Trust, London 


EBP disciplines advocate consideration of patient-specific health characteristics and preferences, clinical expertise and 
resources locally available. These are often presented as descriptive accounts, when research outcomes are expressed in 
measurable numerical data. Bayesian models offer the possibility of compiling and weighing all of these factors, in an 
equation that produces quantifiable probabilities. No such models are routinely used in EBP. The idea is to design a user- 
friendly package with statistician, IT and EBHC expert input. Patient preferences, disease characteristics, local expertise 
and cost would be given a scale value representing “prior probabilities” that, combined with study data, will yield the 
“posterior distribution”. This would be a quantified expression of the individual’s probabilities. This approach would 
promote shared decision-making, make EBP more acceptable within clinical teams and enable more meaningful 
discussions with patients. 


Corresponding author: Achilleos Haris — Imperial College Healthcare NHS Trust — Email: haris.achilleos@doctors.org.uk 


45. Patients’ web self-diagnosis and treatment: from don'ts to dos 
Beltramello Claudio, Pezzato Franco? 
1GIMBE Foundation, 2Vision Consortium 


More and more patients access health services after googling their symptoms, and suggest examinations or even 
diagnoses and treatments. This bothers the clinical staff. Thanks to the huge progress in artificial intelligence, some apps 
are smart enough to compete with an experienced clinician. We suggest exploiting patients’ tendency to use the web for 
their health complaints, and reshaping it within the framework of an Evidence-Based diagnostic & treatment app. Once 
the health care system provides the EB-app, patients should be encouraged (e.g. reductions in user fees, priority access) 
to submit their health condition before seeing the clinical staff. The latter will keep their leading role in guiding the 
patient through the ultimate clinical pathway, but this time moving from standardized EB information instead of 
randomly collected one. Moreover, the app could result pivotal in suggesting behaviors to prevent and manage chronic 
diseases, which in turn should increase compliance. 


Corresponding author: Beltramello Claudio — GIMBE Foundation Faculty — Email: cla.beltramello@gmail.com 
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46. Innovation for clinical rehabilitation research 


Gerber Naomi Lynn?, Cai Cindy’, Garfinkel Steven’, Weinstein Ali? 
4nova Health System, 7American Institute for Research, ?George Mason University 


Systematic reviews (SRs) often exclude clinical rehabilitation research (CCR) studies as inadequate to inform practice, 
because CCR rarely uses randomized designs. Thus, thorough assessments of the utility and relevance of CCR studies for 
outcomes research are rarely performed. Even when they are fully assessed, CCR studies often receive low ratings for 
integrating knowledge with dissemination and utilization (KDU). There are 2 main reasons. First, CCR studies rarely 
include measures that advance knowledge dissemination and utilization as primary outcomes. Second, evaluation tools 
for SRs give great weight to randomized designs. Thus, research from disciplines that rarely use randomized designs is 
often discounted. To have greater impact, CCR studies must choose outcome measures that are more relevant to KDU. 
The designers of SR criteria and tools must also think carefully about the implications of their weight structures for the 
ability of SRs to reach useful conclusions for disciplines that rarely use randomized designs, for ethical or practical 
reasons. 


Corresponding author: Gerber Naomi Lynn - Inova Health System — Email: lynn.gerber@inova.org 


47. Automated, continuously updated, crowd-sourced evidence syntheses 


Martin Janet 
MEDICI Centre, Schulich School of Medicine & Dentistry, Western University, London, Canada 


The constant chase of filtering, extracting, and synthesizing the evidence base to inform clinical questions has spawned a 
significant academic and commercial market for systematic review and meta-analyses (SRMAs). Yet, despite a growing 
army of human resources working on evidence synthesis, there remains a significant lag-time between release of new 
evidence from clinical trials and publication of updated SRMAs. This asynchrony frustrates decision-makers who rely on 
SRMAs to inform which health interventions to adopt and which to forgo. Given the success of automated information- 
gathering accelerated by crowd-sourcing in other sectors, it is clear that the current modus operandi for human-powered 
“manual” evidence synthesis is long overdue for disruptive innovation. This lightning talk will explore why “artificial- 
intelligence enhanced” continuously updated SRMAs (supplemented by crowd-sourced human curation) will soon be our 
new reality, and how it will leapfrog us beyond status quo. 
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48. Evidence reversibility index & evidence sufficiency index 


Martin Janet 
MEDICI Centre, Schulich School of Medicine & Dentistry, Western University, London, Canada 


In our "post-truth world", evidence has lost its lustre and facts don't seem to matter anymore. The current ‘publish or 
perish’ mindset falsely inflates research and publication. Consequently, an overgrown haystack of poor quality research 
makes searching for the needle of truth exceedingly complex. It also gives unscrupulous seekers excess opportunity to 
cherry-pick studies in agreement with their preconceived conclusions. Even when valid research is selected, it will 
represent only a partial ‘truth’, and citizens and scientists alike feel betrayed when subsequent research results in 
contradictions and ‘evidence reversals’. As global leaders of evidence-based decision-making, in order to drive evidence 
demand at the grassroots, we need to raise social ambient understanding of “evidence” and its usefulness for health- 
related decisions. Importantly, we need an intuitive index to convey evidence validity, relevance, and sufficiency 
(“evidence reversibility index’). 


Corresponding author: Martin Janet — Western University, London, Canada — Email: janet.martin@lhsc.on.ca 


49. The most important and effective means of disruption: changing organizations’ culture 


Schmid George 
Centers for Disease Control and Prevention 


Although little explored until the 1990’s, business and management research has repeatedly shown that the single most 
important and effective means of improving productivity is by changing corporate culture. Healthcare and medical 
organizations have unfortunately not recognized the power that culture holds in an organization, nor that culture can be 
changed quickly. A change in culture must be set by leadership and then embraced by all employees. Results can be 
dramatic. Unfortunately, training in many prestigious national and international academic teaching settings, which 
produce many influential leaders, does not highlight the preeminence of culture nor ways to effect it. If we wish to alter 
the values and subsequent conduct of our organizations, we must begin by recognizing the need to change their culture. 
If we value evidence-based medicine (EBM), interventional epidemiology, patient-oriented decision-making, or cost- 
effectiveness, then business research tells us that success comes only from clearly elucidating these organizational 
values, convincing and recruiting employees of and to these values, and institutionalizing them. 


Corresponding author: Schmid George — Centers for Disease Control and Prevention — Email: gps1@cdc.gov 
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50. Clinician dashboards: the time is now! 


Tilson Julie 


University of Southern California 


"Me: How are your patients doing? 

Clinician: Good. | think. 

Me: Did that change you made for patients with diabetes make a difference? 
Clinician: Probably. I’m not sure. 

Me: How is that new EHR the clinic implemented? 

Clinician: A major hassle; too much data to enter for every visit. 

Me: Can you use all that data to track your patient outcomes? 

Clinician: Nope. Not really. 

Me: WHAT?!?! 


We can’t change what we don’t measure. We must develop technologies to convert electronic health record (EHR) data 
into meaningful information for clinicians in real time. Clinicians need a dashboard that allows them to track their 
practice patterns and aggregate patient outcomes on a computer or mobile device. The dashboard should be user- 
friendly, customizable, and inviting. It should allow clinicians to compare their patient outcomes to relevant colleagues 
and to identify which practice patterns are associated with better outcomes. The promise of EHRs has failed providers, it’s 
time to fix this! 


Corresponding author: Tilson Julie — University of Southern California — Email: tilson@usc.edu 
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Gerrity Martha, Harrod Curtis, King Valerie 
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Transitional cares in hematology: a systematic review 
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A call for proper worldwide vaccine side-effect studies 
Havinga Wouter 


The future of evidence in an age of alternative facts 
Hugman Bruce 
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Liu ben, jones elan, thane kyaw 
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To create Evidence Based Medicine Ireland (EBMI) 
O'toole Eve, O'rourke Niamh, Murphy Georgina 


Selecting tools to assist general practitioners in diagnosing and assessing the severity of work- 
related mental health conditions 
Rasekaba Tshepo, Clements Jacinta, Chakraborty Samantha, Brijnath Bianca, Mazza Danielle 


Searching for implementation evidence: identifying commonly used terms in systematic 
reviews of implementation in healthcare 
Rogers Morwenna, Abbott Rebecca, Bethel Alison, Talens-Bau Juan, Thompson-Coon Jo 
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Rubenstein Lisa, Hempel Susanne, Danz Margie, Liu Jodi, Motala Aneesa, Foy Robbie, Lim Yee-Wei, 
Shekelle Paul 


@ GIMBE 58 


EVIDENCE FOR HEALTH 


74. 


75. 


76. 


77. 


78. 


8" International Conference for EBHC Teachers and Developers 


y The ecosystem of evidence 
' ay Connecting generation, synthesis and translation 


Taormina, 25" — 28" October 2017 


If Nothing Changes...Then, WelL..Nothing Changes: An Analysis of Interventions Presented at 
the 2015 Centers for Disease Control and Prevention (CDC) Epidemic Intelligence Service 
Conference and the 2015 Training Program in Epidemiology and Public Health Interventions 
Network (TEPHINET) Conference 

Schmid George, Shirzad Abdul Shirzad, Hussein Sayed 
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51. How to express EBHC: advances bring clearer decisional emphasis 


Alper Brian 


DynaMed Plus, EBSCO Health 


BACKGROUND. 2017 included many multidisciplinary international networked efforts to improve the clarity in concept 
expression for summarizing what we know in forms to support decision-making. Specific efforts varied across domains 
(evidence synthesis, guideline development, systematic appraisal of recommendations, appraisal of performance 
measures, shared decision-making) and groups but have a common thread of frameworks to provide simple expressions 
of complex concepts. 


AIMS. To summarize the most impactful conceptual developments in 2017 that could change how we express EBHC 


METHODS. We consolidated 7 presentations to the Global Evidence Summit 2017 to provide a synthesis of evidence 
conference presentations. 


RESULTS. 1. Certainty of Net Benefit provides a key concept for decision support between Quality of Evidence and 
Strength of Recommendation. 2. A Net Effect Estimate can be produced with a 95% confidence interval in a practical way. 
3. Recommendations for Preference-Sensitive Decisions can be expressed in a way to harmonize clinical practice 
guidelines and shared-decision making. 4. Well-Informed Shared Decision-Making (WISDM) can be demonstrated for 
atrial fibrillation combining best evidence for effects for stroke prevention and bleeding risk, individual risk prediction, 
and individualized decision support in patient-meaningful ways for 8 options. 5. Classification of Consistency across 
Guidelines can be used for a global view on specific recommendations. 6. Appropriateness Criteria for Performance 
Measures have been defined. 7. Checking the Checkboxes has been done by reviewing appropriateness for 65 primary 
care performance measures 


LIMITS. These 7 developments had substantial overlap in groups and individuals participating but are not formalized 
under a single steering group so there is not an overarching consensus as these ideas develop. 


CONCLUSIONS. Substantial progress occurred in 2017 reaching breakthroughs in simple models to express complex but 
important concepts for how we express what we know and how well we know it in ways to help patients, clinicians and 
policy-makers make specific decisions. 
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52. What predicts independent external validation of cardiovascular risk clinical prediction rules? 
Cox proportional hazards regression analyses 


Ban Jong-Wook 


University of Oxford 


BACKGROUND. Performance of clinical prediction rules (CPRs) should be validated by independent researchers unrelated 
to their derivations. Although many cardiovascular risk CPRs have been developed, most have not been externally 
validated. It is not known why some CPRs are externally validated by independent researchers and others are not. 


AIMS. We aimed to assess the probability of having an independent external validation after cardiovascular CPRs are 
developed and evaluate whether features of development, reporting and publication of cardiovascular CPRs are 
associated with having an independent external validation. 


METHODS. We analyzed cardiovascular risk CPRs included in a systematic review. Independent external validations were 
identified by conducting forward citation searches of derivation studies. Time between the publication of derivation study 
and the first independent external validation was calculated for each cardiovascular CPR. We assessed Kaplan-Meier 
estimates of probability to have an independent external validation after the publication of a CPR derivation. Using Cox 
regression, we explored whether 12 characteristics of derivation, reporting, and publication of cardiovascular risk CPRs 
are associated with time to the first independent external validation. 


RESULTS. Of 125 cardiovascular risk CPRs we analyzed, 29 had an independent external validation. The median follow-up 
time was 118 months (95% Cl, 99-130). It took at least 122 months (95% Cl, 91-299) for 25% of cardiovascular risk CPRs to 
have an independent external validation. Cardiovascular risk CPRs from the US were 4.15 times (95% Cl, 1.89-9.13) more 
likely to have an independent external validation. Raising the sample size of derivation by ten times was associated with 
a 2.32-fold (95% Cl, 1.37-3.91) increase in the probability of having an independent external validation. Cardiovascular 
risk CPRs presented with an internal validation tend to get an independent external validation quickly (HR = 1.73, 95% Cl, 
0.77-3.93). Reporting all the information needed for calculating individual risk were 2.65 (95% Cl, 1.01-6.96) times more 
likely to have an independent external validation. Publishing a cardiovascular risk CPR in a journal that has one unit 
higher impact factor was associated with a 6% (95% Cl, 3-9) increase in the incidence of an independent external 
validation. 


LIMITS. Although we attempted to identify all conflicts of interests between authors of derivation and external validation 
studies, some collaborations may not have been traceable. The number of available cardiovascular risk CPRs and the 
independent external validations in our analyses precluded assessing predictors of independent external validation in a 
multivariable analysis. 


CONCLUSIONS. Probability for cardiovascular risk CPRs to get an independent external validation was low even many 
years after their derivations. Authors of new cardiovascular risk CPRs should consider using adequate sample size, 
conducting an internal validation, and reporting all the information needed for risk calculation as these features were 
associated with an independent external validation. 
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53. Search summary tables for systematic review: making the search process more efficient and 
transparent 


Bethel Alison, Abbott Rebecca, Rogers Morwenna 


Evidence Synthesis Team, University of Exeter Medical School 


BACKGROUND. Development of transparent, efficient and effective search strategies is a fundamental cornerstone of 
systematic review methods. Yet, there is little consensus concerning which databases to search, the number of databases 
to search or how to choose them, with most review teams relying on the experience of an information specialist. Authors 
of systematic reviews are recommended to publish their search strategies, databases searched and supplementary 
searching methods, yet most do not routinely publish detailed information on where the final included studies were 
found. Developing a system which helps information specialists and review teams identify key databases for specific 
topics should make future searching for systematic reviews be more efficient and improve transparency of the searching 
process. 


AIMS. We sought to evaluate whether collecting and collating which databases contained the studies that were 
ultimately included in a systematic review into a simple summary table is both feasible and useful for information 
specialists involved with systematic reviews. 


METHODS. We took three systematic reviews that we had published and developed two ‘search summary tables’ for 
them. The first table simply identified which databases and supplementary search methods identified the included 
studies. The second table extended this and identified whether the databases picked up the included studies when the 
search was re-run at the end of the review. We then invited information specialists attending an information specialist 
‘one day workshop’ and an online discussion forum to help pilot the two tables. We asked for feedback on ease of use, 
perceived importance and suggested amendments. 


RESULTS. Completing the search summary tables highlighted for us: i) which databases provided unique studies, ii) which 
combination of core databases provided all the included studies, and iii) which databases provided no studies. The 
extended table revealed the strength of re-running the search at the end of the review, and search efficiencies that can 
be made. For example, in a review of mindfulness which had eight included studies: six originally were identified across 
nine databases and two from supplementary searching. The tables clearly show that all included studies would have 
been retrieved from searching just three databases (Embase, Medline and Psycinfo), and when re-running the searching 
at the end of the review, all eight studies were located in the same three databases. Feedback on the use of the tables 
included suggestions to include a column for format and to include rows to help demonstrate sensitivity and precision. 


LIMITS. We acknowledge that it may take some time to develop a database of which databases are most useful for 
particular topics, but if this became standard practice, an evidence base would soon accumulate. Search summary tables 
require time to complete and access to the review resources. If information specialists are not fully embedded in the 
review team it may be more difficult for them to complete the tables as it does require access to all the search 
information. Reviewers may consider the tables to be too time consuming and reflective, especially if they are not going 
to be doing further reviews. 


CONCLUSIONS. The search summary tables developed and tested are a simple way to collate the search information that 
should be held for a systematic review. If search summary tables were routine and made available as part of a published 
systematic review, It would enhance our knowledge about database selection and supplementary search methods, and 
may result in more efficient, effective and transparent search methods. 


Corresponding author: Abbott Rebecca — University of Exeter — Email: r.a.abbott@ exeter.ac.uk 
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54. Multidimensional assessment of women with ovarian cancer: increasing value of health 
research 


De Vincenzo Francesco, Cammisuli Davide Maria, Innocenti Augusto, Cosentino Chiara, Pruneti Carlo 


Department of Medicine and Surgery, University of Parma, Italy 


BACKGROUND. Beyond physical pain, ovarian cancer in women presents with a range of psychological symptoms. 
Survivors may experience chronic fear of recurrence, anxiety and depressive symptoms, sexual dysfunction and even 
identity disturbance. Screening for psychological distress may be useful to identify treatment compliance and reduce 
negative emotions. 


AIMS. The aim of the present study was to better understand how psychological adaptation to ovarian cancer may 
interfere with the course of the disease and suggest useful guidelines to follow for a tailored psychological intervention. 


METHODS. 22 ovarian cancer survivors (mean age, 55.7 yrs.) underwent a psychological evaluation after disease 
remission at Medical Oncology Unit of Parma Maggiore Hospital. After a clinical interview carried on by hospital 
psychologists, women were tested by a wide psychodiagnostic battery, including: Multidimensional Scale of Perceived 
Social Support (MSPSS), Derriford Appearance Scale (DAS), EORTC Quality of Life Questionnaire 3.0 Version (EORTC QLQ- 
C30), Coping Orientation to Problems Experienced New Italian Version 25 (COPE-NVI-25), Post-Traumatic Growth 
Inventory (PTGI) evaluating social support, body image, quality of life, coping strategies, and positive outcomes reported 
by persons who have experienced traumatic events, respectively. Heart Rate Variability (HRV) has been revealed, too. 
After descriptive statistics, a non-parametric data analysis was performed by Spearman’s rho correlations. 


RESULTS. COPE-NVI-25 positive attitude was positively associated with DAS total score (rho= -0.64, p < .05), social 
awareness (rho= -.53, p < .05), sexuality awareness (rho= -.67, p < .001), Self negative concept (rho= -.64, p < .001), 
awareness of face aspect (rho= -56, p < .006), physical distress (rho = -.45, p < .05). COPE-NVI-25 orientation to problem 
was negatively associated with DAS total score (rho= -49, p < .05), Self negative concept (rho= -.55, p < .05) and awareness 
of face aspect (rho= .52, p < .05). Low and high frequencies of HRV were related to PTGI relation to others (rho = -.57, p< 
.05), personal strength (rho= -.56, p < .05) and new possibilities (rho = -.60, p < .05). 


LIMITS. The present study should be improved by the collection of more extensive data and by a comparison with the 
same measures at baseline. 


CONCLUSIONS. Coping with the aftermath of a trauma might be related to autonomic nervous system someway. An 
alloplastic tendency to deny negative feelings and emotion was evident in women survived ovarian cancer. Future 
research should be addressed to evaluate the effectiveness of this specific strategy for psychological adaptation to the 
disease. 
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55. Engagement and partnership in the conception, design, implementation and evaluation of the 
5As team program 


Campbell-Scherer Denise?, Luig Thea’, Asselin Jodie’, Anderson Robin’, Fielding Sheri?, Borowitz Michelle}, 
Torti Jacqueline?, Giacomoni Penny’, Johnson Jeff?, Sharma Arya 


University of Alberta, University of Alberta, ?University of Lethbridge, 7Edmonton Southside Primary Care Network, Edmonton Southside 
Primary Care Network, University of Alberta, University of Alberta, ‘Member of the Public, University of Alberta, University of Alberta 


BACKGROUND. In this presentation we focus on the participatory action research process of integrating the tacit 
knowledge of people living with obesity and frontline healthcare providers with the research knowledge of an 
interdisciplinary research team. The purpose of this process is to both develop contextually appropriate, sustainable, and 
meaningful interventions that result in multifaceted impact; and, engagement to support trial execution, evaluation, 
dissemination and sustainability. 


AIMS. Obesity is a pressing health concern, however there is recognition that healthcare providers struggle to help 
people living with obesity. The 5As Team program has focused on working with people living with obesity to understand 
what they want from their healthcare team, co-creating with people and frontline healthcare providers resources and 
tools to support collaborative deliberation, and with interdisciplinary primary care teams to change the quality and 
quantity of obesity management in primary care. 


METHODS. The grounding frameworks for the 5AsT Program include the Collaborative Deliberation Model, Theoretical 
Domains Framework, Complex Intervention Implementation, and the Knowledge to Action Cycle. For the assessment of a 
co-created peer supported educational intervention on increasing the quality and quantity of obesity management in 
primary care, we used a randomized control trial with convergent mixed method evaluation. For the patient studies we 
have used qualitative methods with data from interviews, videos, and diaries analyzed with thematic analysis. 


RESULTS. For the randomized control trial, co-creation resulted in a contextually appropriate intervention resulting in 
multifaceted change in provider practice, team practice, clinic and organization operations, with sustained changes in 
training and programming. Lessons learned were that release of control by the researchers was necessary for this to 
occur, and that true engagement resulted in a different way of implementing the trial, which proved crucial. Internal 
practice facilitation by a partner was critical to the success of the intervention. Partnership with people living with 
obesity working with the team to vet ideas and approaches have been crucial to the work to develop the patient 
intervention. 


LIMITS. As with any complex intervention work in real-world settings, context is a major determinant of what is possible 
to do in research. This means that in pragmatic randomized trials it is necessary to negotiate what can be done in the 
research; while trying to optimize internal validity is important, there are trade-offs in achieving increased external 
validity. While qualitative work is not generalizable, studies like this one can result in new insights and principles which 
may inform similar work in other jurisdictions. 


CONCLUSIONS. Working together the 5As Team partners, patients and researchers have created meaningful shifts in the 
confidence and quality of obesity prevention and management in a large primary care network. Ongoing work to co- 
create a tailored, whole person intervention responsive to contextual factors and peoples’ health goals is shedding light 
on ways providers can learn to help people living with obesity. 
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56. The awareness and knowledge of EBM in the international orthopedics community: a 
questionnaire survey 


Castellini Greta+, Gianola Silvia2, De Girolamo Laura’, Denti Matteo?, Seil Romain?, Banfi Giuseppe? 


University of Milan, IRCCS Galeazzi, *Centre Hospitalier Luxembourg-Clinique d'Eich 


BACKGROUND. Evidence Based Medicine (EBM) is a universal cultural movement that integrates three important aspects: 
current best evidence, clinical expertise and patient’s values. EBM’s influence and application is exponentially growing. 
How clinicians perceived and use EBM has been explored in studies across several countries encompassing many medical 
fields. However, no one has studied clinician’s behavior under an international perspective. Indeed, whether the 
international orthopedic community endorse the EBM principles and apply them in their clinical practice is still largely 
unknown. 


AIMS. To investigate the awareness, knowledge and perception towards the helpfulness and usefulness of the evidence 
based literature in clinical practice among surgeons and health professionals, member of the European Society of Sports 
Traumatology, Knee Surgery and Arthroscopy (ESSKA Association), which is a professional society that represents Europe 
in the fields of degenerative joint disease and sports medicine. 


METHODS. A questionnaire has been sent to orthopedic surgeons and health professionals who are members of the 
ESSKA Association during the year 2016 (January — December). Professionals contacts were collected from an electronic 
database settled in Luxembourg. We used the self-report questionnaire of Jette et al. Jette et al. Physical Therapy, 2003), 
which was built to investigate the knowledge of the evidence based medicine among physical therapists, demonstrated 
to have an adequate test-retest reliability. The questionnaire has been adapted for being appropriately addressed to the 
orthopedic community. The modified questionnaire has been judged by an expert panel of four health professionals in 
order to validate the content for the orthopedic community. We explored: (i) respondents’ attitudes and beliefs about 
evidence based medicine; (ii) interest in and motivation to transfer evidence based medicine to practice; (iii) educational 
background, knowledge and skills related to accessing and interpreting information; (iv) level of attention to and use of 
literature; (v) access to and availability of evidence; (vi) perceived barriers to using evidence in clinical practice. The 
following demographic and practice data have been collected: country, age, sex, educational background, attendance to 
EBM courses. The questionnaire has been generated using an online format (https://da.surveymonkey.com/) and send via 
e-mail to all the selected members of the ESSKA association. The survey has been launched on January 23, 2017 and 
lasted until March 30, 2017. Data were analyzed through descriptive statistics and we will further investigate age, 
educational background (i.e. PhD or only MD) and country as co-variates to response to items. 


RESULTS. 288 ESSKA members, 263 males and 25 females, answered to the questionnaire, corresponding to the 10% of 
the total member’s number. The median age was comprised among 40 — 49 years. About the 2.5% of the responders work 
in two countries and the 1.7% even in a third. The majority of responders was a medical doctor (91%), and the minority 
(8%) was represented by other health professionals such as physical therapists, biomedical engineers and sport scientists. 
The 45.5% of our sample has a post-graduate degree and the 30% undertook a PhD. EBM cultural movement is known by 
the 97% of responders and more than 50% of them declared to have received EBM training during the academic 
preparation. However, almost the 50% declared to have not received a specific training to critically appraise the research 
literature. The 90% of the responders agreed that EBM improves the quality of patient care but the 25% declared that the 
EBM does not take into account patient preferences. 


LIMITS. We will investigate the non-response bias due to the size of the sample selected. 


CONCLUSIONS. How clinicians in orthopedics field implement the EBM movement in their practice is paramount to 
discover. Our preliminary results can lead to investigate educational lack and professional behavior in order to improve 
the quality of orthopedic practice. Further actions should be direct towards restraining divergent beliefs, promoting 
education and disseminate EBM strategies. 
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57. Describing a user-centred approach to developing key questions for a clinical guideline for 
general practitioners 


Chakraborty Samantha, Clements Jacinta, Brijnath Bianca, Rasekaba Tshepo, Mazza Danielle 


Monash University 


BACKGROUND. Involving guideline users during the development of a guideline increases the likelihood that the 
guideline will be implemented. Plausibly, this is because an inclusive approach highlights the needs of guideline users 
and the contextual factors that might influence implementation by these users. Our team is creating a clinical guideline 
for general practitioners (GPs) entitled “Clinical guideline for the diagnosis and management of work-related mental 
health conditions in general practice”. GPs are, however, a challenging cohort for guideline developers to influence. One 
way to potentially facilitate eventual implementation of this guideline by GPs is to ensure the guideline addresses key 
questions and concerns of GPs. The Clinical Reasoning Framework is the systematic process of clinical judgement that 
GPs use to investigate and manage symptoms in their patients, and may therefore provide a useful approach to 
understanding GP needs and developing key questions. 


AIMS. To apply a user-centred approach, based on the Clinical Reasoning Framework, to develop key questions for the 
work-related mental health guideline. 


METHODS. International best practice approaches for developing guideline key questions were used to create an 
overarching process for the development of key questions. We then integrated steps to obtain and incorporate the 
clinical and contextual needs of GPs into this processes. This included using the Clinical Reasoning Framework, as the 
analytical framework to organise these needs. 


RESULTS. A seven-step approach to the development of the guideline questions was created. These are: 1) Define the 
rationale for the guideline; 2a) Generate an initial list of key questions based on qualitative interviews with primary end- 
users (GPs) and secondary end-users (psychiatrists and compensation scheme workers), and extend these findings with a 
literature review. 2b) Use the Clinical Reasoning Framework as the analytical approach for assessing qualitative data; 3) 
Define the key questions; 4) List the relevant outcomes for each key question; 5) Review and revise the draft key 
questions; 6) Prioritize draft key questions; and 7) Decide on the final list of key questions. 


LIMITS. The inclusion of qualitative interviews with users requires a significant allocation of time and funding. In addition, 
research and clinical skills that are required to undertake this task warrant the inclusion of expertise in qualitative 
research and clinical practice on a guideline team. 


CONCLUSIONS. A user-centred approach, using the Clinical Reasoning Framework, can be applied as an innovative 
approach to determine guideline key questions for GPs. 
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58. Translating clinical guidelines into performance measurements: the ECIBC example on breast 
cancer 
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Lerda Donata 


European Commission, Joint Research Centre 


BACKGROUND. The European Initiative on Breast Cancer (ECIBC) aims to ensure and improve the quality of breast cancer (BC) 
services across European countries. The overall goal is to contribute to improve health, and reduce health inequalities in 
Europe. The key tools are: (i) development of evidence-based recommendations (the European Breast Guidelines), and 
collecting existing high-quality evidence-based guidelines on all processes of BC care on the Guidelines Platform), and (ii) a 
voluntary European Quality Assurance scheme for BC services (the European QA scheme) to enhance and assess their 
implementation. The working modality is transparent, inclusive and multidisciplinary, considering the needs, preferences and 
values of persons using BC services, and involving them at all stages of the ECIBC’s development and implementation. 


AIMS. Provide insight on how the ECIBC model may effectively combine the development of evidence-based 
recommendations with their implementation through a QA scheme covering the whole BC pathway. 


METHODS. The recommendations in the European Breast Guidelines are formulated by the ECIBC Guidelines 
Development Group (GDG) using GRADE. The questions to be addressed are prioritised from a list prepared by the GDG, 
including stakeholders’ proposals, and evidence is underpinned by existing or newly developed systematic reviews. 
Person-relevant outcomes are included. Evidence-to-Decision frameworks are used to provide a systematic and 
transparent approach for going from evidence to healthcare decisions. The European QA scheme foresees a set of 
requirements, from screening to end-of-life care. To be certified under this scheme, BC services must comply with these. 
Compliance will be evaluated via on-site audits and, when appropriate, also via indicators. Requirements are selected 
applying an adjusted RAND/UCLA-methodology; they are defined from literature, including existing schemes, or new 
ones are developed, if needed. In Delphi-like rounds, these are identified by the ECIBC Quality Assurance Scheme 
Development Group (QASDG), based on understandability and relevance and, thereafter, for technical feasibility. 
Requirements/indicators identified by QASDG will be approved by countries for being implementable in the various 
health systems. Finally they will be tested in a pilot run. The ECIBC working modality is transparent, including a structured 
management of conflict of interest, and GDG and QASDG follow the Commission rules for Expert Groups. JRC scientific 
and technical coordination of ECIBC is independent of all commercial, private and national interests. 


RESULTS. ECIBC brings together a wide range of actors at European level, including patients, professionals, NGOs, 
industry, policy makers, and country representatives. They are involved via public calls at all relevant ECIBC development 
and implementation phases, as already happened with the scopes of the European Breast Guidelines and European QA 
scheme. The guidelines are web-based and specifically tailored for each of three profiles: citizens and patients, health 
professionals, and policy makers, and will be translated into all European languages. The ECIBC model implies that 
requirements of the European QA scheme will be based on recommendations derived from the European Breast 
Guidelines and from existing guidelines (the Guidelines Platform) thus contributing in dissemination and implementation 
of high-quality and evidence-based guidelines for BC. In addition, a long-term monitoring strategy of ECIBC impact is 
planned. The expected benefits of the ECIBC include greater confidence in BC services and reduction of health 
inequalities in Europe. 


LIMITS. With regards to the guidelines, the challenges identified relate to 1) prioritisation of questions, and 2) use of 
indirect evidence and modelling studies or even of good practice statements when direct evidence is scarce or lacking, 3) 
time needed for experts to issue a recommendation. Concerning the QA scheme, these relate to 1) basing requirements 
and indicators on evidence, 2) ensuring they are relevant for persons, and 3) ensuring they are technically feasible in each 
country. Even the most relevant requirements are useless if they cannot be implemented or if they are unable to 
distinguish between acceptable and poor quality of care. 


CONCLUSIONS. Guidelines and QA schemes must be relevant, feasible and implementable to have an impact on 
healthcare and reduce inequalities in countries (for ECIBC more than 30 involved, each with varied resources and 
healthcare systems). ECIBC aims for implementation via (i) a multidisciplinary, transparent and robust development 
process, (ii) continuous involvement of all stakeholders, particularly countries, and (iii) use of the European QA scheme 
also as a tool to disseminate and assess implementation of high quality evidence-based guidelines. If successful, the 
ECIBC will also provide a model transferrable to other cancers and health problems. 
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59. Scoping reviews of research literature to inform rehabilitation clinicians about functional 
outcomes for patients with burn and spinal cord Injury 
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BACKGROUND. Background: The field of clinical rehabilitation research (CRR) is developing criteria to provide unique 
identifiers that provide reliable and useful information that can inform practice. Function, participation and societal 
integration are critical outcomes for CRR. We believe they are under-represented in published CRR literature. 
Additionally, the field has not reached consensus about which outcomes are both reliable, efficient and relevant to 
clinical practice. We report results of two scoping reviews and assess the frequency with which CRR outcomes include 
measures of function, participation or societal integration. Scoping reviews are designed to perform a literature review 
using a predetermined, systematic approach. It accepts a wide variety of study designs into the search in order to 
determine relevant and reliable information about a particular topic. It is often used in emerging fields to assess what has 
been investigated rather than what level of evidence exists for the inquiry and whether research gaps exist. 


AIMS. Aims: To assess the frequency with which CRR outcomes include measures of impairment, function/functional 
limitation, participation or societal integration. Determine the frequency with which specific standardized instruments 
are used in CRR. Perform these reviews in patients with burn injury (BI) and spinal cord injury (SCI). 


METHODS. Methods: We followed the guidelines suggested by Levac in performing a scoping review. We defined the 
questions as stated in the aims. Publication dates were 1990-2016 inclusive. We performed a search in both PubMed and 
Medline for BI and SCI using the following terms: ((traumatic spinal cord injury) AND rehabilitation) AND functional 
outcomes; ((traumatic spinal cord injury) AND (rehabilitation or physical therapy or occupational therapy)) AND 
outcomes AND (function or functional or mobility) and similar for burn injury. Exclusion criteria: not peer reviewed, 
foreign language, non-human, did not measure function or participation. 


RESULTS. Results: Bl: Total number of articles identified were 212. Seventy-four met all inclusion criteria and no exclusion 
criteria. Thirty-one studies were intervention trials and 42 were descriptive studies. There were 35 studies that included 
functional measures and 3 where it was stated to have been measured, but no measures were presented; 12 studies 
measured societal integration and 5 measured participation in life activities. Twenty-six studies measured impairment or 
symptoms exclusively which did not relate these findings to function. The overwhelming majority of these studies were 
performed during hospitalization for acute care or acute rehabilitation. SCI: Total number of articles identified were 545, 
of which 235 were intervention trials and the remainder being descriptive. There were 144 studies that used a standard 
evaluation and 91 that measured impairment exclusively without functional relevance. Studies employed the Functional 
Independence Measure in >60% and walk distance measures in >40%. The reports included both inpatient n=(50) and 
outpatient (n=50) or both (44) status. 


LIMITS. Limitations of Study: One limitation of this review approach is that the search terms are dependent upon what the 
researchers use for keywords. Often the term “rehabilitation” is used as an outcome to include exclusively symptoms such 
as pain, sleep disorder etc; or impairments such as scarring, range of motion or strength; or for quality of life which is only 
generically measured. These may certainly contribute to what we consider function or other outcomes sought for 
rehabilitation interventions, but they may not. In this review, we selected a strict criterion for rehabilitation that required 
evidence of the use of functional or participation measures. Therefore, the number of studies that were selected for 
review were a relatively small number. This is especially true for the BI literature. 


CONCLUSIONS. Conclusions: The scoping reviews reported demonstrate several important issues for future discussion 
concerning knowledge dissemination and translation for CRR. The SCI and BI rehabilitation literature consists of 
relatively few intervention trials. The intervention trials do not always use standardized evaluation tools and this is 
especially true for the BI studies for which there is no generally used metric of function. These types of instruments 
provide important post-acute care information but the majority of studies are performed during hospitalization or at 
discharge for patients with BI and for about half of patients with SCI. To advance our understanding of effectiveness, 
value to patients and relevance for practice, CRR may benefit from addressing these issues. 
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60. Enhancing state policymakers’ ability to use research evidence 
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BACKGROUND. The Reforming States Group, a bipartisan organization of legislative and executive branch leaders from 
most US states, requested assistance educating colleagues about evidence-informed policymaking. The Center for 
Evidence-based Policy at Oregon Health & Science University and Milbank Memorial Fund collaborated to develop a two- 
day Evidence-informed Health Policy (EiHP) workshop. 


AIMS. Build capacity within state governments to use research evidence in health policymaking, particularly for state run 
Medicaid programs that serve socially and economically disadvantaged populations. 


METHODS. Workshop objectives are to: 1) introduce concepts essential to using evidence in policy decisions (e.g. study 
design, risk of bias, relative vs. absolute risk), 2) foster relationships among policymakers and key staff to support 
evidence-informed policymaking, and 3) introduce resources for finding evidence summaries (e.g. Cochrane). For each 
state, a senior official convenes legislative and executive branch members with influential roles and identifies important 
health issues for the state. Workshop faculty use these issues in the workshop to demonstrate practical applications of 
research to policy decisions. Workshop evaluations include ratings of amount learned and benefit vs. time invested. In 
2015, a qualitative evaluation was done of responses to open-ended questions (e.g. what worked, what didn’t, use in 
policymaking) and interviews of workshop faculty. 


RESULTS. Twelve states and 263 policymakers participated in workshops and most completed evaluations. On a scale of 0 
= nothing to 6 = a lot learned, the average was 5.4. Participants felt better equipped to find and use evidence and 
challenge claims made by others. Some found the content too detailed. Two states requested refresher sessions, 3 
requested assistance with evidence resources, and several requested help in communicating research information to 
constituents. Creating workshops of varying lengths (2 hours to 2 days) for different policymaker audiences occurred in 
response to the evaluations. 


LIMITS. Outcomes were limited to self-reported outcomes and did not include objective documentation of impact on 
policy decisions. Policymakers have little time to devote to learning about what constitutes good quality evidence and 
want easy to use checklists and sets of questions they can use. 


CONCLUSIONS. EiHP workshops introduce state policymakers to concepts needed to acquire, appraise, and apply 
evidence and an opportunity to strategize about implementing these processes in their work. 
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61. Rapid reviews to inform state health policy decisions 


Gerrity Martha, Harrod Curtis, King Valerie 


Oregon Health and Science University 


BACKGROUND. Medicaid, a US federal-state partnership, provides care to poor and disabled children and adults. 
Seventeen state Medicaid programs collaborate, as the Medicaid Evidence-based Decisions (MED) project, with the 
Center for Evidence-based Policy (CEBP) to produce rapid reviews (RR) to inform health policy decisions. 


AIMS. To describe the methods used to produce RRs for US state policymakers. 
METHODS. Review of CEBP RR methods, their evolution, current outputs, and feedback from state policymakers. 


RESULTS. Between 2006 and 2016 the CEBP produced over 350 RRs. Initial MED evidence reports (2006-2008) were brief 
evidence summaries of SRs from trusted core sources such as the Cochrane Library. Reports sometimes included searches 
to update SRs, but critical appraisal (CA) of studies was not routine. In 2008, participating policymakers requested more 
complete evidence reviews, and wanted the option to add clinical practice guidelines, with CA of the guidelines, and 
payer coverage policies. Since 2009, CEBP has routinely incorporated the following methodology elements into RR 
production: refinement of topic scope and protocol with nominating state; multiple database search for SRs and update 
searches using PubMed; single reviewer study selection/abstraction/CA, with second reviewer validation; narrative 
synthesis; internal review; and proprietary publication. Optional elements, incorporated based on the topic and needs of 
the nominator, include: peer-reviewed search strategy; search of additional databases; industry dossiers; guidelines and 
payer policies; dual review of selection/abstraction/CA; external review and public dissemination. Production timeline is 
generally three to four months from protocol agreement. 


LIMITS. Outcomes only include self-reported policy changes and their impact following presentation of a report. 
Evaluation of the collaborative process and interacton among states has not be done. 


CONCLUSIONS. Medicaid programs have successfully collaborated with the CEBP over the last decade to develop RRs to 
inform policy decisions. Evidence reviews alone seldom provide sufficient information for good policy formation. 
However, critical analyses of evidence and appraisal of the quality and concordance of guidelines and policies with that 
evidence can assist policymakers in making evidence-informed decisions within relevant timeframes and available 
resources. 
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62. Multidisciplinary biopsychosocial rehabilitation for chronic low back pain: the need to present 
minimal important differences units in meta-analyses 
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BACKGROUND. Results of meta-analyses are often difficult to be interpreted: the relevance in clinical practice is often 
elusive. 


AIMS. Our purpose is to report meta-analyses in terms of minimal important difference (MID) units to better understand 
results in clinical practice; then we aim to compare findings of meta-analyses reported as clinical and statistical 
significance difference. 


METHODS. We re-analyzed the data of a Cochrane review focusing on multidisciplinary biopsychosocial rehabilitation 
(MBR) since it is one of the most used treatment for low back pain. We first searched for all pain instruments having an 
anchor based MID. Imputation of MID was adopted for instruments without an established MID. We considered MBR 
versus usual care for chronic low back pain in short, medium and long terms. The outcome of interest was pain. We 
express the results in MID units: if the overall effect size is greater than 1 many patients will gain clinically important 
benefits from treatment, if it lies between 0.5 and 1.0 an appreciable number of patients will benefit, and if it falls below 
0.5 MID units only a little number of patients will achieve important benefits. 


RESULTS. When compared to usual care, in short and medium term MBR improves back pain in an appreciable number of 
patients: the MID is lower but close to 1. In Longer time MBR probably makes Little or no effect for the majority of patients: 
in fact the MID approaches 0. There are few plausible explanations: effects decreased in magnitude over time; true 
efficacy is more limited and early findings are biased or spurious eventually. 


LIMITS. Meta-analyses in MID units are vulnerable to unexperienced, oversimplified interpretation since the MID is 
subjective. 


CONCLUSIONS. Meta-analyses expressed in MID units offer better insights about the clinical relevance of MBR. 
Multidisciplinary Biopsychosocial Rehabilitation, even if related to a statistically significant advantage at all follow-up 
times, has only a clinically modest effect that can affect the actual recommendations on the use of multidisciplinary 
biopsychological rehabilitation, especially in the long term. 
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63. Usefullness of the TRIPOD guideline to a prediction model development improvement 
(PREDICE-SCORE) 


Gomez De La Camara Agustin, Lora Pablos David, Menendez Orenga Miguel 
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BACKGROUND. in 2012 our group developed a clinical prediction rule in “de novo” heart failure (HF) called PREDICE- 
SCORE. The predictive final selected variables were age, dependency for daily basic activities, creatinine clearance, 
sodium levels at admission and systolic dysfunction diagnosis. Subsequently, the TRIPOD guideline was published. The 
TRIPOD Statement offers a series of recommendations to improve the reporting of predictive models studies as articles.. 


AIMS. The aim of this project was to assess the quality of PREDICE-SCORE development process and locate possible 
deficiencies, following the TRIPOD’ recommendations 


METHODS. TRIPOD Checklist was used to evaluate PREDICE-SCORE degree of development quality.. 


RESULTS. In the original article, several of the recommendations of the TRIPOD guideline were not attended: those 
related to measures of accuracy used in the construction of the statistical model and in the clinical prediction rule; the 
management of lost data; and the unappropriated examples for the use of the predictive model. In addition, we detected 
two mistakes in the original manuscript: Table 3 heading n=412 instead of n=600 and Table 4 with errors in the 
transcription of the percentages that estimate the risk. 


LIMITS. It could be some self-complacency in eveluating owns wark, that we try to overcome with and extra self-criticism. 


CONCLUSIONS. We found missing and wrong information in the PREDICE-SCORE original published article that could 
have been avoided. TRIPOD recommendations follow up clearly contribute to the improvement of prognostic 
research.TRIPOD guideline is an useful tool to evaluate and certify quality elements that should contain predictive 
models, both in development and validation. 


Corresponding author. Gomez De La Camara Agustin — Hospital Universitario 12 de Octubre — Email: acamara@h12o0.es 


&@ GIMBE 7” 


EVIDENCE FOR HEALTH 


The ecosystem of evidence 
wy Connecting generation, synthesis and translation 


64. Transitional cares in hematology: a systematic review 
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BACKGROUND. Transitional cares are constituted by a set of actions with the goal of ensure coordination and continuity 
of health care during the transition of a patient between different health settings. The aim of this model of care is to 
improve the outcomes of subjects discharged from the hospital with an elevated risk of re-hospitalization with an 
elevated resource consuming pattern of care. Several studies concerning outcomes of onco-hematological patients after 
discharge from the hospital for antineoplastic treatments have been published to define access criteria, feasibility and 
safety of domiciliary care during the chemotherapy induced neutropenia. 


AIMS. To evaluate the impact of transitional care programs on re-hospitalization of discharged hematological to home 
from published randomized controlled trials. 


METHODS. Five databases have been searched and gray-literature. Results were screened to identify RCT articles about 
adults admitted for treatment of hematological malignancies and discharged to home with comprehensive care- 
programs or gold standard procedure. No limits of publication time have been considered. 


RESULTS. Electronic research has done between October and December 2016. This systematic review followed the 
Cochrane process and was guided by use of PRISMA statements. No studies have been found that would respect fully the 
inclusion criteria, but two studies are considered for their relevancy. The studies demonstrate that early discharge is safe, 
feasible and efficient in hematologic patient with malignancy, but percentage of readmissions after early discharge was 
not the primary outcome of the studies. 


LIMITS. Only studies published in English were searched. 


CONCLUSIONS. Although many studies on transitional care are reported, we found little evidence by published RCT about 
the potential efficacy of a comprehensive transitional care model to reduce the hospital readmission in onco- 
hematological patients. 


Corresponding author: Guarinoni Milena Giovanna — Brescia Hospital Spedali Civili- Email: 
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65. A call for proper worldwide vaccine side-effect studies 


Havinga Wouter 


GP NHS 


BACKGROUND. Has, due to vaccinations and immunisations, the global burden of disease (GBD) increased from (acute) 
infectious diseases to a greater burden of disease due to (chronic) non-communicable diseases (NCD)? 


AIMS. Advocating for evidence, by calling for proper vaccine side-effect studies particulary in view of the intent to subject 
every person on the planet to vaccines and immunisations. It is essential that, with that intent, there is a plethora of 
ongoing studies that check whether the endemics of non-communicable diseases that have erupted in the developed 
world are not related to the introduction of vaccines and that this possible adverse effect is not replicated in the 
developing countries due to the initiation of immunisation programs there. 


METHODS. Personal observation of the literature over the past 20 years and raising awareness in the EBM community 
that proper vaccine - and immunisation side-effect studies are almost non-existant. 


RESULTS. Proper ongoing vaccine-side effect studies are missing, which gives the impression that vaccine acceptance is 
based on "evidence-based hearsay". 


LIMITS. Differing belief systems, each generating their own supportive evidence. 


CONCLUSIONS. Vaccination and immunisation is the one remaining bastion where "eminence" based medicine seems to 
rule. It is dominated by the Public Health school of though which believes in the panacea of vaccines and immunisations. 
Any dissenting opinions that go counter this stance are neglected and apparently actively suppressed as there are still no 
proper ongoing studies into vaccine side-effects. 1 The current situation gives the impression that vaccine acceptance is 
based on "evidence-based hearsay" generated by the Public Health school of thought. "Understanding the science of 
hearsay may be useful to help salvage evidence-based medicine." 2 To answer the question whether there is an increase 
in the GBD due to the introduction of vaccines, proper worldwide vaccine side-effect studies are needed 1 1) Re: Is the 
timing of recommended childhood vaccines evidence based? Havinga W. http://www.bmj.com/content/352/bmj.i867/rr-7 
2) Does evidence-based hearsay determine the use of medical treatments? John PA. loannidis 
http://dx.doi.org/10.1016/j.socscimed.2017.02.004 
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66. The future of evidence in an age of alternative facts 


Hugman Bruce 


Uppsala Monitoring Centre 


BACKGROUND. The effective presentation of medicines safety information for patients has been taken increasingly 
seriously in recent years. However, little attention has been paid to the knowledge and skills needed for engaging with 
those who are disenchanted or hostile with regard to scientific evidence in the health setting. The emergence of 
‘alternative facts,’ the politicisation of data, threats to the credibility of specialist expertise, science-denial, and what has 
been called the ‘post-truth’ era, all present significant challenges which have not been extensively addressed. 


AIMS. The aims of the presentation are (1) to offer insight into the nature of the threats to the credibility and influence of 
scientific evidence and (2) to make some suggestions as to what processes might facilitate dialogue or rapprochement. 


METHODS. This project is part of my continuing intellectual research in the field of risk communication (this includes, 
amongst other topics, published work on risk communication and medicines for women and the impact of bureaucracy 
on patient safety). The current work is at a productive, early stage, with material of sufficient strength to share with 
international colleagues. 


RESULTS. Adherence to positions or practices that are hostile to scientific evidence (vaccine-doubters, seekers of 
homeopathic and some alternative therapies, climate sceptics, for example) is shaped and driven by multiple and 
variable complex influences. These are amenable to few, if any, of the normal processes of rational, scientific discourse or 
to evidence as exemplified in the concept of EBM or even in some wider definitions. Obtaining a hearing, let alone 
productive dialogue, demands exceptionally subtle and empathetic approaches that most scientists and authorities are 
ill-equipped to undertake. A rare example of success was the eventual overcoming of resistance to the vaccination of 
children against polio in northern states of Nigeria in the past decade. The lessons of that crisis, along with more recent 
thinking and practice, continue to provide pointers for managing current challenges. 


LIMITS. This work represents an intellectual, philosophical enterprise, with assumptions and insights as yet untested in 
the real world. It is the essential precursor to further thinking, research and practical work. 


CONCLUSIONS. More data, better data, improved presentation of evidence are, largely, irrelevant to efforts to reduce the 
health and other risks to which science-sceptics are vulnerable. A subtle understanding of their sometimes complex or 
obscure motivation and beliefs is essential, and novel, agile and experimental communications are needed if any kind of 
meeting of minds or rapprochement is to be possible. 


Corresponding author: Hugman Bruce - Uppsala Monitoring Centre - Email: brucehugman@hotmail.com 


tt] Gl M BE 75 


EVIDENCE FOR HEALTH 


The ecosystem of evidence 
wy Connecting generation, synthesis and translation 


67. Differences in post-operative bowel function and ileus between right and left sided radical 
colorectal resections 


Liu Ben, Jones Elan, Thane Kyaw 


Glan Clwyd Hospital, Betsi Cadwaladr University Health Board, UK 


BACKGROUND. Enhanced Recovery (ERAS) programme recommends early commencement of post-operative diet. Is this 
equally feasible among different types of colorectal resections? The ERAS society has warned that early feeding can lead 
to increased risk of vomiting, and have identified several factors that influence post operative ileus e.g. laparoscopic 
surgery, mid-thoracic epidurals instead of systemic opioids etc. At Glan Clwyd Hospital, we suspected that ileus is more 
common among radical right-sided colonic resections than left-sided equivalents, but very limited literature has hitherto 
addressed this. 


AIMS. This retrospective comparative study aims to answer the question: are there differences in post-operative bowel 
functions and the rates of post-operative ileus between right and left-sided radical colonic resections? 


METHODS. Data from 141 radical colorectal resections from a total of 171 resections overall performed during a two- 
year period (from 2014 to 2016) are divided into two groups: 77 cases of right-sided colectomies (right and extended-right 
hemicolectomies) and 64 cases of-left sided resections (left hemicolectomies and anterior resections). The inclusion 
criteria are: both emergency and elective surgeries; patients with either primary bowel anastomoses or stoma formations. 
The exclusion criteria are non-radical resection (e.g. for diverticular disease or inflammatory bowel diseases). Patient 
characteristics, operative techniques, time to passage of flatus and motions, tolerance of diet, incidence and duration of 
post-operative vomiting and overall hospital stay are compared. We also take into account of whether patients have 
received prokinetics or laxatives which are potential sources of bias. Statistical analyses used are: T-test for parametric 
data, Mann-Whitney U test for non-parametric data, and Chi-Square test for nominal data. 


RESULTS. Except for the rates of stoma formation, which is higher in the left-sided colectomies, no statistically significant 
differences in the baseline characteristics were observed (i.e. patient age, gender, laparoscopic/open surgery, 
emergency/elective procedures). Patients undergoing right colonic resections take longer to establish bowel motion 
(mean of 4.92 days vs. 4.00 days; p=0.028), tolerate diet (mean of 6.03 days vs. 3.42 days; p = 0.017), and vomit for longer 
(median of 5 days vs. 2.5 days; p = 0.029) compared with left-sided resections. These findings are statistically significant on 
two-tailed statistical analyses. To minimise bias, we use subgroup analyses for patients with primary anastomoses only 
(given the different rates of stomas between the two groups) and those having elective surgery only. The differences 
between left and right colonic resections remain statistically significant for both time to passage of bowel motion and 
tolerance of diet. The overall hospital stay for patients undergoing right-sided surgery is longer (mean of 11.34 days) than 
left sided colonic surgery (mean of 7.57 days; p = 0.031). There are no statistically significant differences in time to pass 
flatus, incidence of vomiting or clinical/radiological documentation of paralytic ileus. There are no statistically significant 
differences in the usage of laxatives or prokinetics. 


LIMITS. Our research is a retrospective study, which is subject to a certain degree of potential bias. A randomised 
controlled trial is obviously not feasible. Our unit has however embarked on collecting data prospectively since the 
completion of this study. There are potential sources of bias that are yet accounted for, such as the usage of systemic 
opioids, length of surgery, and the amount of blood loss during surgery. Although our hospital has an established ERAS 
programme, the length of recovery in terms of opening bowels and tolerating diet after surgery appears to be greater 
than described in other literature. Perhaps the adherence to the ERAS protocol may be partially accountable for this. An 
audit is needed to measure our compliance. 


CONCLUSIONS. This study demonstrated a slower recovery of bowel functions in patients undergoing radical right-sided 
colonic resection compared to left-sided surgery. We propose that the slower bowel function found in radical right-sided 
colonic resections is an effect of disturbances of nervous tissues around the duodenum during right colonic mobilisation. 
This is in contrast to early experimental data suggesting left colonic anastomosis being the determining factor for the 
recovery of bowel function, in an era where radical lymph node dissection was not routine. This may explain rates of post- 
op ileus in D3-lymphadenectomies being as high as 9%. Perhaps ERAS should be modified for right-colectomies. 


Corresponding author: Liu Ben — Glan Clwyd Hospital — Email: jpbenliu@yahoo.co.uk 
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68. Does knowledge brokering improve the quality of rapid review proposals? A before-after study 


Moore Gabriel?, Redman Sally, D'este Catherine’, Makkar Steve’, Turner Tari? 


1The Sax Institute, 7The Australian National University, 7Monash University 


BACKGROUND. Rapid reviews are increasingly being used to help policy makers access research in short time frames. A 
clear articulation of the review’s purpose, questions, scope, methods and reporting format is thought to improve the 
quality and generalisability of review findings. 


AIMS. The aim of the study was to explore the effectiveness of knowledge brokering in improving the perceived clarity of 
rapid review proposals from the perspective of potential reviewers.To conduct the study, we drew on the Evidence Check 
rapid review program, where policy makers draft a review proposal (a pre knowledge brokering proposal) and have a 1- 
hour session with a knowledge broker, who re-drafts the proposal based on the discussion (a post knowledge brokering 
proposal). 


METHODS. We asked 30 reviewers who had previously undertaken Evidence Check reviews to examine the quality of 60 
pre and 60 post knowledge brokering proposals. Reviewers were blind to whether the review proposals they received 
were pre or post knowledge brokering. Using a six-point Likert scale, reviewers scored six questions examining clarity of 
information about the review’s purpose, questions, scope, method and format and reviewers’ confidence that they could 
meet policy makers’ needs. Each reviewer was allocated two pre and two post knowledge brokering proposals, randomly 
ordered, from the 60 reviews, ensuring no reviewer received a pre and post knowledge brokering proposal from the same 
review. 


RESULTS. The results showed that knowledge brokering significantly improved the scores for all six questions addressing 
the perceived clarity of the review proposal and confidence in meeting policy makers’ needs; with average changes of 
0.68 to 1.23 from pre to post across the six domains. 


LIMITS. Evidence Check uses a standardised process that may be different from those of other rapid review programs and 
knowledge brokering may differentially affect those review proposals. It is possible that some reviewers were able to 
guess whether they received a pre or post proposal, though it is not clear that this would result in a significant response 
bias. The reported changes in clarity may or may not result in more timely, relevant or useful reports for policy makers 
and there may be other factors at work that were not identified in our study. 


CONCLUSIONS. This study found that knowledge brokering increased the perceived clarity of information provided in 
Evidence Check rapid review proposals and the confidence of reviewers that they could meet policy makers’ needs. 
Further research is needed to identify how the knowledge brokering process achieves these improvements and to test the 
applicability of the findings in other rapid review programs. 


Corresponding author: Moore Gabriel - The Sax Institute — Email: gabriel: moore@saxinstitute.org.au 
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69. EBPsteps: a mobile application to support the process of EBP 


Olsen Nina Rydland?, Johnson Susanne Gradem’, Titlestad Kristine Berg’, Hole Grete Oline’, Frisk Bente?, 
Larun Lillebeth’, Heldal Ilona’, Helgesen Carsten’, Ringheim Johannes Mario}, Fahlvik Morten? 


Western Norway University of Applied Sciences, 7Norwegian Institute of Public Health 


BACKGROUND. Evidence-based practice (EBP) involves five steps: ask, search, appraise, integrate and evaluate. Following 
the EBP steps is regarded as one of several core competences in health- and social care education, and there is an 
international consensus that health- and social care professionals should integrate the knowledge, skills and attitudes of 
EBP into their undergraduate education, life-long learning and patient care. However, students within health- and social 
care programs lack evidence-based practice (EBP) competence. In particularly, they struggle to use EBP and transfer 
evidence to real clinical situations. To address this problem we developed a mobile application (app) called “the 
EBPsteps” at the Western Norway University of Applied Sciences (HVL). Mobile-learning technology has the potential to 
become a central tool for facilitating the learning and use of EBP in higher education, and among post-graduates and 
clinicians. The purpose of the EBPsteps is to support users through the process of EBP by guiding them through the five 
steps. 


AIMS. To describe the development and the content and of the EBPsteps app, designed to support the process of 
evidence-based practice. 


METHODS. The EBPSteps has been through a longer development phase that started in 2015. A multi-professional group 
of researchers took part in the development: physiotherapists, nurses, occupational therapist, social educators and 
engineers with ICT competence. Testing of the app was done both among students who participated as users in the 
development process, and participants at a workshop at a national conference on EBP. 


RESULTS. The EBPSteps app guides users through all of the five EBP steps, all linked to a Norwegian internet-based 
learning resource/course in EBP (www.kunnskapsbasertpraksis.no), provided by The Centre for Evidence-based Practice 
(HVL) and The Norwegian Knowledge Centre for the Health Services (NOKC). This course consists of video lectures, text 
modules and assignments, and is aimed at clinicians, teachers and students in medicine or health professions. In addition, 
the app links to information resources at The Norwegian Electronic Health Library. The library provides free access to 
point-of-care tools, guidelines, systematic reviews, scientific journals and a wide variety of other full-text resources. 
Example of other support is a glossary of key terms related to EBP. Users of the EBPsteps can document and save 
information related to each EBP step in the app, and they can easily share this via e-mail. Consequently, the EBPSteps app 
also becomes a system for communication, feedback and formative assessment. The information recorded and stored in 
the EBPSteps app can be used to assess different elements related to EBP competence: knowledge (e.g. type of question 
being asked, or sources of information used); skills (e.g. how to write a precise question or how to critically appraise a 
research article); and behaviour (e.g. changes in practice and involvement of patients in clinical decisions). The app can 
be freely accessed at a website (https://ebpsteps.no/) and used via any type of smartphone, computer or iPad (using the 
Google Chrome browser). 


LIMITS. The EBPSteps app resembles other available apps and tools, including the CASP app and the Graphic Appraisal 
Tool for Epidemiological studies (GATE). A major difference is that the EBPSteps app provides links to learning resources 
for each EBP step, including links to sources of evidence. However, EBPsteps is still a beta version, and further 
development is needed through exploring user experience and performance, and determining the potential of the app to 
support learning and practicing EBP. The EBPSteps app can easily be translated to other languages than Norwegian and 
made available to other countries, including low- and middle-income countries. 


CONCLUSIONS. The EBPsteps has the potential support the process of evidence-based practice among students and 
clinicians. This app guides users through all of the five EBP steps and provides links to learning resources for each step, 
including links to sources of evidence. In addition, the EBPsteps can be used to assess EBP competence. 


Corresponding author: Olsen Nina Rydland — Western Norway University of Applied Sciences — 
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70. To create Evidence Based Medicine Ireland (EBM) 


O'toole Eve’, O'rourke Niamh?, Murphy Georgina? 


1National Cancer Control Programme, Ireland, Department of Health, Ireland, 7Naji Foundation United, Kingdom 


BACKGROUND. The EBM movement was established over twenty years ago. It originated in both Oxford and McMaster 
university in Canada with the unifying force in both being Dr David Sackett. Whilst the approach to putting evidence into 
practice has grown throughout the world its adoption in Ireland has been somewhat fragmented and its practice is still 
often seen as a paradigm shift. There are practitioners throughout Ireland but no unifying network to connect the 
practitioners, educators and students of EBM 


AIMS. To establish a hub for evidence based practice in Ireland (EBMI) and to promote evidence based practice 
throughout the healthcare system in Ireland with the ultimate goal of improving patient outcomes 


METHODS. « To create hub for EBM in Ireland (EBMI) * To establish a network of EBM practitioners * To establish a 
network of EBM educators * To identify a core curriculum for EBM for undergraduate and post-graduate health 
professionals in Ireland * To create international links and access to world class experts in EBM 


RESULTS. There are two goals for 2017 A three day workshop on EBM is being developed. This shall be delivered in 
collaboration with the CEBM in November 2017. This will be used to establish a network of both EBM practitioners and 
educators in Ireland. Implementing evidence based change or teaching of EBM shall be a goal set for all participants. The 
introduction of EBM into the curriculum for undergraduate and postgraduate healthcare professionals is being 
facilitated. Baseline research on current education and training in EBM in Ireland is currently being carried out to bench 
mark against international standards and to set a national core curriculum for EBM. 


LIMITS. This project requires ongoing commitment from those involved. This change project also relies on a curtural 
acceptance within the health services executive in Ireland to suppor tand promote the use of evidence based practice. 
This initiative requires funding for training and education of key personel in EBM. It is currently supported by the Naji 
foundation and it will require sustained funding to achieve its long term objectives. 


CONCLUSIONS. This project will address the current lack of standardised EBM education, training and practice in Ireland 
at all levels. The value to the target group of healthcare professionals is — availability of training in Ireland, accessibility 
in terms of cost and convenience, delivery from experts in the field, training alongside a multidisciplinary group of health 
professionals working in the Irish health system and building a future network of EBM advocates. There will be a focus on 
quality, sustainability, identifying change agents and on implementation of evidence into practice. The establishment of 
EBMI will commence with the development of a network of practitioners and educators identified through the EBM 
workshop. EBMI will provide a hub for evidence based research, education, collaboration and dissemination and will 
maintain strong links with the international EBM community. 


Corresponding author: O'Toole Eve — National Cancer Control Programme — Email: eve.otoole@cancercontroLie 
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71. Selecting tools to assist general practitioners in diagnosing and assessing the severity of work- 
related mental health conditions 


Rasekaba Tshepo, Clements Jacinta, Chakraborty Samantha, Brijnath Bianca, Mazza Danielle 


Monash University, Australia 


BACKGROUND. Claims for work-related mental health conditions (MHCs) are increasing in Australia. In Australia, general 
practitioners (GPs) see approximately 96% of injured workers, and play a key role in facilitating recovery for people with 
work-related MHCs. However, GPs have expressed difficulties with diagnosing and managing work-related MHCs. This is 
being addressed by the development of a “Clinical Guideline for the Diagnosis and Management of Work-Related Mental 
Health Conditions in General Practice.” A key question to be addressed in this guideline is whether tools exist to assist in 
diagnosing such conditions. 


AIMS. To identify clinical assessment tools that can be used by GPs to facilitate accurate diagnosis of work-related MHCs 
and their severity. 


METHODS. We conducted a systematic search of the literature in MEDLINE, EMBASE, PsycINFO, and CINAHL, using a 
combination of MeSH terms and keywords associated with “work-related MHCs”, “assessment tools”, “sensitivity and 
specificity” and “general practice”. The search was limited to papers published in English, with no set date limit. To assess 
the quality of studies that evaluated tool sensitivity and specificity, we used the Quality Assessment of Diagnostic 
Accuracy Studies (QUADAS) checklist. Identified tools were also evaluated using criteria for patient-based outcomes: 
appropriateness, reliability, validity, responsiveness, precision, interpretability, acceptability and feasibility. Case 
screening and diagnostic classifications data (true and false positives, true and false negatives, and/or receiver operator 


characteristics-area under the curve) were extracted for analysis of sensitivity and specificity. 


RESULTS. Sixty assessment tools were identified from 38 studies. Twenty-three tools were not relevant to work-related 
MHCs and were therefore excluded. Thirty-five of the remaining 37 tools had not undergone sensitivity and specificity 
testing in the work-related MHC context. Two tools, from three studies, had undergone diagnostic accuracy testing and 
had sensitivity and specificity data. The first tool was the Patient Health Related Questionnaire-9 Item (PHQ-9) in work- 
related depression, with the MINI International Neuropsychiatric Interview being used as the reference standard. The 
PHQ-9 had a sensitivity of 64.3% to 93.9% and a specificity of 71% to 87%. The second tool was the Post-Traumatic Stress 
Disorder (PTSD) Checklist-Civilian version (PCL-C), with the Diagnostic and Statistical Manual-lV being used as the 
reference standard. The PCL-C had a sensitivity of: 69% (mild PTSD), 90% (moderate PTSD) and 82% (severe PTSD). 
Specificity for the PCL-C was 97% (mild PTSD), 79% (moderate PTSD) and 67% (severe PTSD). 


LIMITS. There are few studies that identify clinical assessment tools in the work-related MHC and general practice. Even 
fewer of these tools have undergone sensitivity and specificity evaluations in the work-related MHC context. This limits 
the evidence-base to inform guideline recommendations for the diagnosis of MHCs and their severity in this context. 


CONCLUSIONS. Two tools, the PHQ-9 and PCL-C for PTSD, have been assessed for sensitivity and specificity to diagnose 
MHCs in the work-related context and have the potential to assist GPs to diagnose work-related MHCs in practice. 
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72. Searching for implementation evidence: identifying commonly used terms in systematic 
reviews of implementation in healthcare 


Rogers Morwenna, Abbott Rebecca, Bethel Alison, Talens-Bau Juan, Thompson-Coon Jo 


University of Exeter Medical School 


BACKGROUND. In recent years there has been a rapid growth in the amount of published implementation research. 
Locating studies of implementation research in healthcare is challenging as there is large variation in terminology 
amongst authors and implementation experts and there can be disagreement about definitions of implementation itself. 
Researchers who are trying to understand what helps and hinders implementation need more efficient ways of 
identifying implementation literature. In previous work we assessed the sensitivity of specific terms for implementation 
across three reviews of implementation (in care homes, in dementia, and a review of reviews in healthcare). We found 
that terms for methods of implementing change (e.g. experts, audit and feedback, educational workshops) retrieved more 
relevant records than terms describing the process (e.g. ‘bench to bedside’, ‘knowledge mobilisation’ and ‘knowledge 
translation’). 


AIMS. The aim of this study was to test the terms that we found to have the best sensitivity in previous work against 
systematic review abstracts retrieved from the Canadian Agency for Drugs and Technology in Health (CADTH) Rx for 
Change database. The Rx for Change database contains current research evidence about intervention strategies used to 
alter behaviours of health technology prescribing, practice, and use. 


METHODS. Systematic reviews held on the CADTH Rx for Change database were selected by quality rating and checked 
for inclusion on MEDLINE. Search terms and phrases for implementation identified were searched in the title and abstract 
fields of the systematic reviews on MEDLINE. Sensitivity of the terms was examined and compared with previous results. 


RESULTS. There were 932 systematic reviews included on the CADTH Rx for Change database categorised by 
professional, organisational, consumer, financial and regulatory interventions. There were 189 systematic reviews with an 
AMSTAR rating of 9-11, which formed the test set. Analysis of the title and abstracts indicated that some search terms and 
medical subject headings were more effective than others in retrieving systematic reviews in implementation. 


LIMITS. Rx for Change is a database specific for evidence related to changing health behaviours which may have 
impacted the sensitivity of implementation terms. Further testing across other resources/databases is required. 


CONCLUSIONS. The findings will aid researchers and information specialists designing searches to retrieve evidence on 
implementation. 


Corresponding author: Abbott Rebecca — University of Exeter — Email: r.a.abbott@ exeter.ac.uk 
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73. Promoting knowledge transfer for quality improvement interventions: a validated literature 
search, screening, and critical appraisal method 


Rubenstein Lisa?, Hempel Susanne, Danz Margie’, Liu Jodi, Motala Aneesa?, Foy Robbie’, Lim Yee-Weit, 
Shekelle Paul? 


RAND Corporation Santa Monica, *University of Leeds 


BACKGROUND. Effective learning across related scientific investigations through evidence synthesis is critical for 
promoting evidence-based approaches to healthcare. The role of local quality improvement interventions (Qlls) in 
changing healthcare is now widely recognized, and frequently published. Knowledge transfer for findings from QlIl 
publications, however, poses challenges. Evidence reviews specifically directed at QlIl’s are hampered by search and 
review challenges not addressed by standard evidence review methods. For example, the article selection process must 
enable selection of work directed toward local sites, rather aimed at generalizability. On the other hand, if the quality of 
reporting is too low, little that is worthwhile can be learned. Excellent QI reporting standards exist; these however, are 
designed for future publications and can be aspirational and comprehensive. Currently usable critical appraisal tools for 
Qlls, in contrast, must overcome current ambiguities in QI language, be widely applicable, and be focused on the most 
important evaluation standards. They must also be reliable in the hands of evidence reviewers as well as valid. 


AIMS. We aimed to develop and validate 1) a QII search and screening strategy; and 2) a critical appraisal instrument (the 
Quality Improvement Minimum Quality Criteria Set or Ql-MQCS). 


METHODS. We carried out a multi-year process for QII search, screening, and quality criteria development and validation. 
The process built on existing reporting guidelines and included a nine person international expert panel and workgroup 
effort characterized by three pre-panel online surveys, iterative pilot data collection, six expert panel telephone calls, 
and two in-person panel meetings. The final search strategy combined QI, continuous QI, QI intervention components, 
and EPOC-eligible intervention search terms. Two reviewers title- and abstract- screened the first 1/3 of articles resulting 
from applying the search strategy in PubMed. We used the title search to generate a machine learning algorithm to 
screen the remaining articles. Finally, we applied our QIl screening algorithm to include studies focusing on healthcare 
delivery organizations; reporting on QIl effectiveness, impacts or success; reporting outcomes on patients, caregivers, 
provider behavior or process of care; and aiming to change how delivery of care is routinely structured. In a parallel 
process, we iteratively developed and validated our critical appraisal criteria with our expert panel. The final QI-MQCS 
includes the 16 domains: organizational motivation, intervention rationale, intervention description, organizational 
characteristics, implementation, study design, comparator, data source, timing, adherence-fidelity, health outcomes, 
organizational readiness, penetration/reach, sustainability, spread, and limitations. Two reviewers independently 
assessed each article using the tool. We analyzed item endorsement (number meeting the criterion); interrater reliability 
(Kappa, percent agreement), and internal consistency (inter-item correlations, conceptual overlap). 


RESULTS. We published on the validity of our search algorithms for identifying QIl literature and the validity and 
reliability of our algorithm-based screening tools. We applied the QI-MQCS to 54 diverse QIl evaluation publications 
chosen using our published search and screening strategies. Selected studies were from developed and developing 
countries and addressed a wide range of QI problems. The median inter-rater agreement on the 16 QI-MQCS items was 
kappa 0.52 and 82% agreement. Sources of disagreement included primarily reviewer omissions, a problem for QIl 
literature because of non-standard information placement in articles. Remaining disagreements were due to differences 
in interpretation of the article or criterion. Criteria were statistically conceptually independent. The percent of articles 
meeting each of the 16 criteria ranged from 44% to 93% with a median of 67%, indicating ability to differentiate high and 
low quality studies. 


LIMITS. The QI-MQCS does not judge study design, given existing design review criteria and the diversity of Qll literature. 
Also, iterative testing resulted in elimination of some potentially important concepts that could not be reliably assessed. 


CONCLUSIONS. QII literature is abundant but may not support optimal knowledge transfer without reliable, valid search 
and screening strategies and application of quality review criteria. The QI-MQCS is the result of an international 
workgroup process to develop a valid, reliable, and practical critical appraisal tool for use in QIl evidence review. 


Corresponding author: Rubenstein Lisa - RAND Corporation — Email: rubenstein15501@ucla.edu 
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74. lf Nothing Changes...Then, WelL..Nothing Changes: An Analysis of Interventions Presented at 
the 2015 Centers for Disease Control and Prevention (CDC) Epidemic Intelligence Service 
Conference and the 2015 Training Program in Epidemiology and Public Health Interventions 
Network (TEPHINET) Conference 


Schmid George’, Shirzad Abdul Shirzad?, Hussein Sayed? 


1Centers for Disease Control and Prevention, 7Afghanistan Ministry of Public Health 


BACKGROUND. The best evidence to advance change comes from effective, and cost-effective, interventions. American, 
European and international training programs in epidemiology have, despite use of the word “Intervention” in many program 
names, focused on observational epidemiology. Fields of health services research that concentrate on interventions such as 
randomized controlled trials or operations research are seldom taught. This is a mistake; unless interventions are performed, 
health and healthcare are not improved. We believe that, typically, investigations confirm that a problem exists and then state 
that an intervention is needed (often using the word “urgently”). Far too often, these interventions are not done. 


AIMS. To confirm our suspicions: 1) that interventional epidemiology is seldom taught, we reviewed the training programs 
of the American Centers for Disease Control and Prevention (CDC) Epidemic Intelligence Service (EIS) and the 
international Training Programs in Epidemiology and Public Health Interventions Network (TEPHINET). TEPHINET is a 
professional network of 67 field epidemiology training programs (FETPs) in more than 100 countries which provides 
training in epidemiology to FETPs as part of its mandate; and, 2) that interventional epidemiology is seldom practiced, we 
analyzed the oral presentations of the 2015 EIS Conference (2015 EISC) and the 2015 TEPHINET Conference (2015 
TEPHINETC) at which CDC and FETP scientists report investigations to determine how frequently interventions were 
performed and reported. 


METHODS. 1) We reviewed materials from the EIS and TEPHINET training programs; 2) three reviewers read and judged 
abstracts. Two reviewers read the 2015 EISC and two the 2015 TEPHINETC, with one reviewer reviewing both. Abstracts 
were judged to have had an intervention or not and, if so, whether the principal intention was to present an intervention 
(“primary intervention’) or, if the intervention followed a noninterventional epidemiology investigation, a “secondary 
intervention.” Abstracts were then graded by whether the presenters called for an intervention to be done or not 
(“intervention needed”). Discrepancy in scoring was resolved by discussion. 


RESULTS. 1) The training programs of neither CDC nor TEPHINET contained modules for the training of operations 
research nor interventional epidemiology (randomized controlled trials or quasi-experimental research). 2) Neither book 
of abstracts contained the words “randomized controlled trial,” “operations research” or “operational research.” The 2015 
TEPHINETC contained 126 presentations, of which 28 (22.2%) contained an intervention (three primary, 2.4%; 25 
secondary, 19.8%). Eighty-five abstracts (67.5%) called for an intervention to be done. The 2015 EISC contained 103 
presentations but we have been able to read to date only 61. Of these, 11 (18.0%) contained an intervention (three 
primary, 4.9%; eight secondary, 13.1%). Thirty abstracts (49.2%) called for an intervention to be done. In both books of 
abstracts, costs were seldom mentioned. We shall review the abstracts of the 2015 European Scientific Conference on 
Infectious Disease Epidemiology (ESCAIDE) in a similar way and, using 2016 abstract books of all three conferences, 
determine if interventions from any observational epidemiology investigation in the 2015 books were reported, although 
high numbers are unlikely given the paucity of primary interventions in the EISC and TEPHINETC abstract books. 


LIMITS. We have investigated only epidemiology conferences, which may not be representative of all scientific disciplines 
which might report or advocate for change. 


CONCLUSIONS. CDC and TEPHINET do not emphasize operations research nor interventional epidemiology. While the 
need for an intervention is frequently and appropriately highlighted by CDC and FETP authors performing observational 
epidemiology, it is unclear who will do (and evaluate) those interventions or even if the interventions will be done. We 
believe CDC EIS and FETP training programs should be revised to emphasize and institutionalize interventional 
epidemiology. Without evidence-based interventions, and which consider costs, healthcare programs cannot make 
rational decisions on what interventions to implement to improve health. 
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75. Unpublished trials in molluscum contagiosum: a case study in publication bias 


Van Der Wouden Johannes 


VUmc University Medical Center Amsterdam 


BACKGROUND. Molluscum contagiosum is a benign skin disorder of viral origin, often seen in children. In our Cochrane 
review ‘Interventions for cutaneous molluscum contagiosum’, we included all interventions studied in randomized 
controlled trials. One of the treatments that was included was imiquimod cream, a drug enhancing the immune response. 
In a previous version of the review (2009), two small studies were included that had imiquimod in one of the treatment 
arms. Only 41 patients had been given the treatment. The study comparing imiquimod cream to its vehicle (placebo) 
(n=23) found a risk ratio (RR) of 3.67 for our primary outcome, complete clearance of lesions 4 weeks after end of 
treatment, with a huge confidence interval: 95% Cl 0.48 to 28.00, including unity (no difference). The paper stated that 
imiquimod was "well-tolerated and effective". In 2014, we were made aware of the existence of 3 vehicle-controlled 
imiquimod trials in molluscum patients, performed in 2005 by 3M Pharmaceuticals. The only publication that was 
available, was an executive summary produced by the US Food and Drugs Administration. 


AIMS. We aimed to obtain the reports of these 3 unpublished trials and combine the results with the study we previously 
included. 


METHODS. We attempted to obtain the reports of these 3 unpublished trials: contacting the Cochrane Skin Group; 
principal investigators of the trials; and the current proprietor of the drug. Results for complete clearance after 4 weeks 
and side effects were pooled using a random effects model with Cochrane Review Manager software. 


RESULTS. The current proprietor kindly provided us with the full reports of the studies. All three were performed in 
pediatric patients, with a total of 827 patients, 532 of whom were assigned to imiquimod cream. Adding these studies to 
the one we already had, changed the effect to RR 0.93, and the confidence interval to 0.61 to 2.26, a reduction in size by > 
94%. Side effects were reported more often in the imiquimod group than in the placebo group: RR 1.41 (95Cl 1.13 to 1.7) 
for all application site reactions, and RR 4.33 (95%Cl 1.16 to 16.19) for severe application site reactions. The company has 
no plans for publishing these studies. 


LIMITS. We can never be certain about the existence of additional unpublished trials. 


CONCLUSIONS. Adding the three unpublished trials changed the conclusion from ‘well-tolerated and effective’ into 'not 
well-tolerated and ineffective’. Many children have probably been exposed to painful and ineffective treatment as a 
result of lack of publication. Exclusion of unpublished trials may grossly distort our view of reality. We should try hard to 
obtain unpublished trial reports. 


Corresponding author: Van Der Wouden Johannes — University of Amsterdam — Email: j.vanderwouden@vumc.nl 
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76. Russian Society of Evidence Based Medicine: 15 years of education 


Vlassov Vasiliy’, Baschinski Saveli? 


National research university Higher School of Economics, Media Sphera Publishing group 


BACKGROUND. 15 years ago a group of physicians and educators from Moscow and S. Petersburg created and in 2003 
formally registered the Society Of Specialists in Evidence Based Medicine (OSDM in Russian, osdm.org). There was an 
intention for the first time in that country — to connect specialists with different educational backgrounds with the aim to 
develop the EBM and evidence-based public health in Russia. 


AIMS. To describe the experience and outcomes of the 15 years of activity 
METHODS. Review of personal collections 


RESULTS. During first years, OSDM coordinated its activities with the Russian branch of the Nordic Cochrane Centre. 
OSDM is represented by branches in the most regions of Russia. From the beginning the society enjoyed the cooperation 
with and influence in the post-Soviet space. In addition to publication of educational materials and provision of 
educational courses, society had succeed in many important developments of the Russian science. To name the major 
ones: - Society lobbied for the open publication of dissertations, and the Government regulation had been changed. Build 
on the open access to dissertations, a new project - Dissernet.ru was created by Russian scientists. As a result, thousands 
of plagiarists were exposed, hundreds were stripped off the degrees. - Society promoted the development of the 
evidence-based guidelines, provided trainings, drafted guidelines for developers. Nowadays, health care quality 
assurance system is under transformation, based on the EB-clinical guidelines. - Teaching basics of clinical epidemiology 
and/or EBM is accepted in almost all medical schools in Russia. - Designs of medical and public health research was 
improved, as well as the use of statistical methods; indirectly the education of physicians in the critical reading has led to 
the improvement of the researchers’ training and the quality of research. - Critical reading lessons, teaching editors and 
promotion of publication standards together with attention paid by Government to the publication activity of researchers 
and quality of publications led to the rising of the journal standards and critical evaluation of the publication practices. 


LIMITS. Unfortunately, Russian practices in the field of education of medical doctors and improving the research 
standards did not hit the targets we set for ourselves. The quality of clinical guidelines is relatively low. The teaching of 
epidemiology in most medical schools is old-fashion infection diseases only. Most medical journals do not employ best 
publication practices yet. Both authors served their terms as presidents of the Society. 


CONCLUSIONS. Society Of Specialists in Evidence Based Medicine was an in-time endeavor with continuing influence on 
the development of multiple aspects of health care and public health on the post-Soviet space. 
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77. Medicines wastage and its association with adherence, beliefs about medicines and locus of 
control in patients suffering from chronic conditions 


West Lorna Marie?, Borg Theuma Ruth?, Cordina Maria? 


University of Malta, *Mater Dei Hospital 


BACKGROUND. Non-adherence is considered to be one of the factors leading to medication wastage. Whilst the literature 
supports the influence of patients’ beliefs on non-adherence, the influence of beliefs on medicines wastage is lacking. 


AIMS. To determine adherence, awareness, beliefs and behaviours regarding medicines and their relationship to 
medicines wastage in patients suffering from chronic conditions. 


METHODS. A structured questionnaire was developed based on a previous questionnaire used amongst the general 
public in Malta and distributed amongst patients having a confirmed diagnosis of asthma, cardiovascular conditions or 
diabetes attending out-patients’ clinics. The first sequential 300 patients who accepted to participate completed a self- 
administered questionnaire which sought to determine awareness about medicines wastage using a 5-point Likert scale, 
adherence to medicines using the ‘Tool for Adherence Behaviour Screening’ (TABS), health locus of control using ‘The 
Multidimensional Health Locus of Control Scale’ Form C (MHLOC) and beliefs about medicines using the ‘Belief about 
Medicines Questionnaire’ (BMQ) Specific and General respectively. The Chi-square test was used to analyse the 
relationship between two continuous variables with a p < 0.05. A logistic regression was performed to ascertain the 
effects of MHLOC, BMQ and demographics in relation to incidence of unused medicines. P-values = 0.05 were considered 
significant. Ethical approval was obtained. 


RESULTS. A total of 300 patients were recruited (58% male; mean + standard deviation age: 61415 years): 100 asthma, 100 
diabetes and 100 cardiovascular. Three quarters agreed that they were fully aware of the issue of medicines wastage 
(74%) and half of participants (49%) agreed that they fill their prescriptions for chronic medicines regularly, whether they 
have run out of them or not. Thirty-nine percent of respondents claimed to have unused medicines at home. More than 
three quarters claimed that no one gave them information on how to store medicines (76%) or how to dispose 
appropriately of them (79%); with only 9% of participants disposing of unused medicines through a medicines disposal 
site. The vast majority of respondents (82%) had suboptimal adherence to medicines, with a significant association 
between non-adherence and unused medicines (?2=7.014, p = .008). The logistic regression model for MHLOC and BMQ in 
relation to incidence of unused medicines was statistically significant, ?2(9) = 54.201, p = .000. Lower ‘general benefit’ 
beliefs (p = .028) and lower beliefs on ‘doctor beliefs’ (p = .001), ‘internal beliefs’ (p = .035) and ‘chance beliefs’ (p = .021) 
sub-scales of the MHLOC were associated with an increased likelihood of having unused medicines, whilst higher ‘other 
people beliefs’ (p = .000) was significantly associated with unused medicines. The male gender (p = .038) and the lower 
level of education (p = .005) were associated with an increased likelihood of a higher incidence of unused medicines. 


LIMITS. While participants seem to be similar to other populations, caution should be exercised in extrapolating the 
results to other communities in view of the differences in healthcare systems, practices and cultures. 


CONCLUSIONS. Incidence of unused medicines is strongly associated with lower beliefs that the medicines are of benefit. 
Therefore, interventions targeting medicines wastage need to recognise these beliefs to influence a change in wastage 


behaviours. 
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i! GIMBE a6 


EVIDENCE FOR HEALTH 


The ecosystem of evidence 
wy Connecting generation, synthesis and translation 


78. Challenges facing non-indigenous researchers undertaking research in Aboriginal and Torres 
Strait Islander communities 


Williams Imelda, Baird Marilyn 


Monash University 


BACKGROUND. Undertaking culturally appropriate research processes in an indigenous setting produces unique 
challenges to non-indigenous researchers due to its complex nature. There is a need to understand and respect 
Indigenous social and cultural values to eliminate misunderstandings. Non-indigenous researchers need to reflect on 
power relationships which compels researchers to engage in a dialogue with leaders from the Indigenous communities. 
For a novice non-indigenous researcher, the concept of cultural safety is new yet necessary to minimise risks that could 
disempower already marginalised communities. Conveying information requires adapted instructions about verbal or 
written communication when engaging with Indigenous communities such as Aboriginal and Torres Strait Islander 
people. Adopting a communication style to include the concept of ‘yarning’ is crucial when e.g. needing to gain consent 
from Aboriginal and Torres Strait Islander participants. Yarning is conducive in creating cultural security for research that 
has as its target population Indigenous communities as it cuts across formal research formalities. 


AIMS. To gain an understanding of the cultural processes when undertaking research in an indigenous setting such as 
Aboriginal and Torres Strait IsLander communities. 


METHODS. The methodology requires both an inquiry and immersion process. To demonstrate adequate community 
engagement, a request needs to be made to the cultural community leaders, in this case the mayors from the local 
community councils, for permission to visit their indigenous communities. Communication techniques such as the 
provision of an explanatory statement and consent form needs to be adapted to reflect culturally safe terminology and 
appropriate medical yarning which is an Indigenous cultural form of conversation. Research conventions applicable to 
the research process must be made visible in the ‘yarn’ to maintain research rigour. In addition the core principles of 
spirit and integrity, reciprocity, cultural respect and safety, equality, responsibility, and survival and protection needs to 
be addressed throughout the research process as it emphasises the inter-relationship of these ethical values and their 
importance to Aboriginal identity. 


RESULTS. Gaps in this novice non-indigenous researcher knowledge emerged which indicates that there should be a 
focus on education towards culturally safe research practices. Training strategies need to equip non-indigenous 
researchers with skills to adopt creative ways of undertaking research in a complex, cultural environment. The complex 
reality of negotiating community-based support to undertake research must also be addressed. In addition there is a need 
to adopt a communication style to include the use of culturally safe terminology and appropriate medical yarning. 
Yarning is useful in establishing a relationship of trust with Indigenous participants. Researcher accountability demands a 
demonstration of respect and acknowledgement of the contribution of Aboriginal or Torres Strait Islander participants to 
the research process. In achieving this, communication to the elders i.e. the community council leaders, about the 
outcomes of the research is essential to demonstrate in particular the core values of spirit and integrity. 


LIMITS. Obtaining ethical approval when targeting indigenous populations is challenging yet transformative. 


CONCLUSIONS. Understanding the cultural processes when wishing to undertake research within Aboriginal and Torres 
Strait Islander communities is crucial for non-indigenous researcher to ensure that cultural safety and cultural respect is 
maintained in these vulnerable communities. Immersion in the cultural processes requires an appreciation for the six 
core ethical values that lie at the heart of Aboriginal and Torres Strait Islander people. For non-indigenous researchers 
the research process undertaken is transformative in a quest towards ensuring cultural safety in marginalised 
communities and at the same time achieving the goals of the intended research. 
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79. Managing Competing Interests While Reviewing The New Employee Orientation Programme, 
at Ahmadi Hospital, Kuwait Oil Company 


Abdel Rahman fatma El Zahraa 


Kuwait Oil Company, Ahmadi Hospital 


BACKGROUND. Orientation to a new job offers the staff member the basic vital information about their Healthcare 
Orgnaisation, their position, the target population/patients & the community;. It should best take place before the job 
starts; i.e. saving more time on the long run; helping the new staff member to know what they are supposed to do & 
knowing how to navigate the organisation. Orientation programme is coherent, combining information, experiences, & 
transmitting the values & culture of the organisation. 


AIMS. Purpose of this Case Report is to share challenges faced, success stories experienced by Ahmadi Hospital - Training 
Teams resolving out possible conflict of interests shown by new employees preferences, organisational culture & 
management strategies, etc. towards developing the «New Employee Orientation Programme»; meanwhile motivating 
our new staff members towards building a positive start with sound foundations to do their jobs more effectively & 
efficiently, integrating themseleves into Ahmadi Hospital workforce & the community. 


METHODS. Ahmadi Hospital has started ealier an optional «Familarization Programme» as short individual visits to 
Division & Units, including Hospital Tour, since 2011. On the occasion for New Ahmadi Hospital inauguration (around mid 
2017) & while updating the pertinent clinical practice guidelines our clinical teams proposed reviewing the current 
relevant strategies, by creating a Mandatory Two-Day «General New Employee Orientation Programme»; covering 
«Multidisciplinary Topics», (i.e. Short Talks). New Employee registers to attend this Two-Day Programme, followed by 
Hospital Tour; & ending by filling an «Online Overall Satisfaction Survey». 


RESULTS. Quarterly Sessions were planned for 2016-2017, all accomplished with high attendance, started by July, 2016 
(102 attendees), Oct, 2016 (32 attendees), Jan 2017 (34 attendees), March, 2017 (plan in progress). «Orientation 
Programme Manuals» are in progress, to be completed & uploaded on Ahmadi Hospital web page, by Sept, 2017. 
«Multidiscplinary Clinical Orientation Programmes» are subject for future updates as well. 


LIMITS. Online «Orientation Manuals» upload will be finalised by September 2017. 


CONCLUSIONS. Three «New Employee Orientation Programmes» (General) was completed successfully for 168 
employees, 2016-2017. Online Satisfaction Survey Analysis, 2016-2017 in progress. 
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80. Assessing the prognostic power of prediction rules: A Users’ Guide to the Medical Literature 


Alba Ana Carolina’, Agoritsas Thomas’, Walsh Michael?, Hanna Steven’, lorio Alfonso?, Devereaux Phillip J?, 
Mcginn Thomas’, Guyatt Gordon? 


University Health Network, *University Hospitals of Geneva, McMaster University, “Hofstra Northwell School of Medicine 


BACKGROUND. Accurate prognostic information is fundamental to optimal clinical care. Patients and clinicians can use 
their intuition and average risk from observational studies to estimate prognosis. The best approach to assess patient 
prognosis, however, relies on models that simultaneously consider a number of prognostic factors and provide an 
estimate of patients’ absolute risk of an event. Such predictive models or rules should be characterized by adequate 
discrimination -differentiating patients who will have an event from those who will not - and adequate calibration - 
ensuring accurate prediction of absolute risk. 


AIMS. We have developed a Users’ Guide to understanding the available metrics for assessing model performance. We 
discussed advantages and disadvantages of current metrics to assess discrimination, including the area under the 
receiving operator curve and c-statistic; calibration, through comparison of observed and predicted risks; evaluated using 
tests (i.e. Hosmer-Lemeshow test) or graphically. We also discussed limitations and advantages of evaluating the relative 
performance of two different models or a modified existing model using risk reclassification analysis. 


METHODS. This presentation used real-world and hypothetical examples to demonstrate concepts and apply results from 
studies evaluating predictive models. This guide gives significant importance to the interpretation of results regarding 
model performance to translate them to the clinical setting. 


RESULTS. Predictive models should be characterized by adequate discrimination differentiating patients who will have an 
event from those who will not, and adequate calibration ensuring accurate prediction of absolute risk. Discrimination 
refers to how well the model differentiates those at higher risk of having an event from those at lower risk. There are a 
number ways to assess discrimination but for binary (eg., dead or alive) outcomes, discrimination is typically 
characterized using the receiver operating characteristic (ROC) curve or, for time-to-event (survival) outcomes, by the c- 
statistic. If the model cannot discriminate between which patients have or do not have an event, the c-statistic (or ROC) is 
0.5 - that is, no better than flipping a coin. If the model always produces a higher probability for patients having events 
versus those who do not have events, the c-statistic is 1.0. Discrimination depends on the distribution of covariates in the 
population in which the model is being used. A model could well discriminate patients with events from those without 
events in a heterogenous population with widely different values of covariates included in the model; however, the same 
model could fail to discriminate a more homogenous population sample. Discrimination alone is, however, insufficient to 
assess a model's predictive capability. However, clinicians should not consider a model that fails to differentiate between 
those with higher risk and those with lower risk. When discrimination is very poor, there is no need to further assess other 
model characteristics. The second necessary model property informs clinicians how similar the predicted absolute risk is 
to the true (observed) risk in groups of patients classified in different risk strata. Calibration refers to the accuracy of 
absolute risk estimates. Poorly calibrated models will under or over-estimate the outcome of interest. Assessing 
calibration involves comparing predicted and observed risk at different levels: in the whole population or mean 
calibration, in different groups of patients based on predicted risk or in different groups of patients based on all possible 
combinations of predictors or covariates. Not uncommonly, more than one model exists to predict an outcome in a 
similar group of patients. Previously developed metrics to compare models, such as the Akaike index criterion (AIC) and 
Bayesian index criterion (BIC), provide statistics that rate models as better or worse but provide no indication of how 
close the best model is to the observed risk. Statisticians have developed new approaches to compare the performance 
of two models addressing the same clinical issue. The most widely used of these approaches compares how individual 
patients are classified in two competing systems: risk reclassification analysis. Results of risk reclassification analysis 
should be report using absolute net reclassification index. 


LIMITS. This guide complements an existing Users’ Guide that addresses the development and validation of predictive 
models. There are some developing metrics (i.e. relative utility or standardized net benefit) providing a formal 
quantification after specifying the relative disutility of false positive and false negative misclassifications. They are not 
yet in wide use, and further discussion is beyond the scope of this manuscript. 


CONCLUSIONS. Together, these guides will allow clinicians to make optimal use of existing prediction models. 
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81. Promoting professional behaviour change in surgical healthcare: how to homogenize paths of 
the hospitals in Local Health Unit TO3. 


Alesina Marta?, Minniti Davide’, Passi Stefano, Rebora Monica?, Siliquini Roberta’, Bellina Maurizio2, Boraso 
Flavio? 


University of Turin, 7ASL TO 3, Piedmont, Italy 


BACKGROUND. Improving quality and effectiveness of healthcare is one of the priority objectives of every health system. 
Finding effective ways to encourage health professionals to routinely embed high-quality clinical evidence into their 
work is important, even if translating research evidence into routine clinical practice is notoriously difficult. In order to 
improve the response to the health needs of the patients with digestive, urological, gynecological problems treated with 
laparoscopic surgery, the Diagnostic Therapeutic Care Paths need to be uniform and homogeneous within the different 
hospitals of the Local Health Unit (ASL TO3). 


AIMS. Aim of the intervention is to homogenize paths of laparoscopic surgery within the hospitals of Local Health Unit 
TO3 in order to act clinical appropriateness and improve outcomes of interventions performed with laparoscopic surgery. 


METHODS. Laparoscopic Surgery Unity creation was planned to optimize surgical paths and its organization, to obtain a 
reduction of intraoperative complications and length of stay, an improvement of surgical outcome and a better use of 
resources. 


RESULTS. Three working groups were identified according to nosological area: digestive, urological, gynaecological. Each 
group has a coordinator responsible for the daily management and coordination, the monitoring of work progress, the 
communication with the referents and the monitoring of the technical and scientific quality of interventions. The program 
is structured in 4 distinct phases: 1) involvement of the working groups to share project objectives, to identify indicators 
and to plan protocols and checklists; 2) sharing protocols and pathways in meeting, peer mentoring and staff meetings; 3) 
control of indications, discussion of cases; 4) checking the objectives and indicators and sharing of project results. 


LIMITS. In order to ensure the homogeneity of all paths it would be necessary to standardize all Diagnostic Therapeutic 
Care Paths and not just those related to laparoscopic surgery. 


CONCLUSIONS. The reinforcement of surgical performance response capabilities related to digestive, urological and 
gynaecological diseases is expected because of the sharing of clinical guidelines, Diagnostic Therapeutic Care Paths, 
clinical risk management protocols, clinical management. This interventions ensure evidence-based medicine approach, 
learning through peer mentoring, multidisciplinary collaborations. 
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82. Comprehensive educational needs assessment of EBP in health policymakers and managers 


Alizadeh Mahasti, Garjani Mehraveh 


Tabriz university of medical sciences 


BACKGROUND. Many barriers impede EIPM, of which dearth of qualified personal is of great importance. Moreover, there 
is no well-established consensus over the content and duration of EIPM courses 


AIMS. To perform a comprehensive educational needs assessment to provide postgraduate tutors with a valid base to 
design effective EIPM courses 


METHODS. We developed a 22-item questionnaire assessing health system directors’ perception of EIPM on a 5-point 
Likert scale. Validity and reliability of the questionnaire was analyzed prior to administration. Delphi survey consisted of a 
list of 43 educational topics and 11 teaching approaches suitable for EIPM courses. Panelists scored the importance of 
each item. In the second round, we sent a summary of results to panelists and asked them to re-score in view of the 
group’s responses 


RESULTS. Of the 95 questionnaires, 63 were responded. This study shows that most of health mangers had highly 
welcoming attitudes toward EIP. However, they did not use research evidence in their decision making. The participants 
had good understanding of technical terms in EIP. Conversely, they had very low access to the relevant databases. We 
observed a strong positive correlation between EIP and awareness of information sources. 9 of 11 EIP experts completed 
the second round of Delphi survey and achieved consensus on twenty educational topics and two teaching methods. 
‘Introduction to EIP’, ‘acquiring evidence’, ‘critical appraisal’, ‘translation of evidence to policy’, ‘assessing the policy’, and 
‘communication skills’ were the categories considered for EIP courses. ‘Group discussion’ and ‘Q & A sessions’ were two 
teaching methods that solidly achieved consensus 


LIMITS. The limited number of participants was the main limitation of this study 


CONCLUSIONS. We recommend postgraduate tutors to simplify the content of EIPM courses, and have more emphasis on 
promoting awareness of relevant information sources. Also, highly interactive teaching methods which address individual 
needs of participants should be employed. 
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83. Use of EUnetHTA tools to implement a local HTA-based decision making process 


Ballini Luciana, Bonvicini Laura, Giorgi Rossi Paolo, Vicentini Massimo 


IRCCS Reggio Emilia 


BACKGROUND. In order to increase transparency of the assessment process and encourage health technologies’ 
developers in submitting assessment requests supported by comprehensive and relevant information, the European 
Network for Health Technology Assessment (EUnetHTA) has developed submission templates for medical devices and 
pharmaceuticals. A similar approach in handling physicians’ requests for innovative medical devices, could help to 
increase the transparency of assessment, appraisal and decision-making processes in Local Health Trusts and Hospitals 


AIMS. To facilitate engagement of health professionals in the Health Technology Assessment process and increase 
legitimacy of the process outputs, a system for technology requests’ submission, based on the EUnetHTA tool, was put in 
place and tested in the Reggio Emilia Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS). 


METHODS. The “EUnetHTA medical devices evidence submission template” for companiesl was adapted for use by 
professionals wishing to propose an innovative health technology for acquisition. Adaptation consisted mainly in 
requiring a more detailed description of the health problem, the current management of the condition and the definition 
of the unmet needs, substantiated by local data. Claimed additional benefits of the technology and potentials for 
additional evidence generation were emphasized. Details on costs and financial resources were also requested. A 
summary with a pre-defined set of brief statements was prepared to guide the decision-making discussion. The headings 
for the 9 one-paragraph statements were as follows: relevance of the health problem; degree of innovativeness of the 
technology; potential clinical impact; potential research relevance; comparative safety and effectiveness; economic 
impact; organizational impact; availability and quality of scientific literature; degree of diffusion. Decision makers 
discussed the statements with the proponents before reaching a conclusion. 


RESULTS. The new process for assessment and appraisal was presented, discussed and approved by the Institution’s board 
of clinical directors. As from 2016 requests for technologies were only examined if presented through the submission 
template. Examples of submissions of three innovative technologies for prostate cancer treatment, endovascular 
procedures and cataract surgery will be presented. Acceptability of submission template was high and professionals were 
successful in completing the document, obtaining support from other expertise available in their institutions (clinical 
engineers, epidemiologists etc.). Decisions, taken within few weeks from submission, resulted in either rejection or 
conditional approval. 


LIMITS. For the process to become feasible and acceptable, proponents were supported by methodologists 


CONCLUSIONS. The EUnetHTA tool proved flexible and valuable in triggering a novel decision-making process. Decisions 
— informed by the 9 bullet statements - were reached with the full involvement of the proponents. Initial professionals’ 
resistance was overcome by transparency and objectivity. References 1) EUnetHTA Evidence Submission Template for 
Medical Device (available at http://www.eunethta.eu/outputs/submission-template-pharmaceuticals-and-submission- 
template-medical-devices ) 
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84. Immunization knowledge and professional attitude assessment of health workers , a specific 
analysis , among occupational 


Casagranda Franca?, Ricco Matteo, Brioni Alessandro’, De Rinaldis Maria Francesca* 


1APSS Trento, 7AULSS 9 Scaligera 


BACKGROUND. Infectious diseases prevention is one of the main goals of Public Health. Some infectious diseases have 
the characteristic of being preventable by vaccination . Health care workers come into contact with patients and 
potentially infected material are at particular risk for exposure to vaccine preventable infectious diseases. Healthcare 
Workers Immunizations Schedule can reduce substantially the number of susceptible health workers and avoid both the 
occupational infections, and the transmission of preventable pathogens from health care worker to patient, or from 
patient to health care worker . The occupational physician is therefore responsible for carrying out all activities linked 
with the prevention and treatment of sicknesses and injuries at the workplace including assessment of immunization 
compliance and vaccination of health care workers. 


AIMS. To evaluate knowledge and healthcare professional attitude towards vaccinations, especially in occupational 
physicians. 


METHODS. Realization of a structured questionnaires, drafted and fitted on specificity set up by Italian NHS and PNPV 
2012-2014 (Itallan National Immunization Program), which aims to investigate general knowledge on vaccines, 
understanding factors influencing vaccination acceptance and risk perceptions towards three pathologies (flu, measles 
and pneumonia). The questionnaire was administered by telephone to 50 occupational physicians randomly selected 
from among the participants of training courses organized by Trento APSS (Trento Provincial health Care Trust) in 
October 2015. For this cross-sectional study, 51 case controls are randomely selected from among Trento APSS health 
workers. Each evaluation has been traced back to a synthetic score with normalized value included between 0 (lower 
knowledge) and 1 (optimal knowledge). 


RESULTS. Occupational physicians (average age 49,6 + 8,0 years, 60% M, 40% W) showed a greater knowledge of 
vaccinations recommended by PNPV 2012-2014 (0,751 + 0,108 vs 0,669 + 0,189 p = 0,0035), and general recommendations 
on immunization practice (0,620 + 0,151 vs 0,512 + 0,205 p = 0,0081). However, knowledge about indication regarding 
vaccinations against measles, mumps, rubella and chickenpox were inadequate. Not negligible percentage of 
occupational physicians (12%) believed true the correlation between vaccinations and autism, and between vaccinations 
and multiple sclerosis. Vaccination acceptance (on average higher in occupational physicians than in health workers), 
was significantly related to level of knowledge in the staff., predictive value, confirmed by the regression analysis. 


LIMITS. The low sample size. 


CONCLUSIONS. The results of the study pointed out as immunization knowledges, from a general perspective to a 
particular perspective (considering specific recommendations required by PNPV to health workers), are significantly 
related to vaccination acceptance.. It is confirmed that the application of an accurate training programs can have a 
positive impact on occupational physicians regarding immunization practice, and on health care workers regarding 
vaccination coverage. 
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85. Evidence-Based Decision-Making for Assisted Reproductive Technologies in Kazakhstan 


Chegedekova Sholpan’, Hismetova Zaituna?, Almadiyeva Alima’, Turgambayeva Assiya’, Inoue Ken? 


1Semey State Medical Univercity, Semey, Kazakhstan, *Medical University, Astana, Kazakhstan, ?Gunma University Graduate School of 
Medicine, Gunma, Japan 


BACKGROUND. The treatment of infertility through assisted reproduction technology (ART) is currently helping hundreds 
of thousands of people each year to achieve parenthood. This branch of medicine is also undergoing substantial 
technological development. While the technology is implemented quickly, solid clinical data become available slowly, 
resulting in a growing gap between current ART practices and evidence-based standards.Measures to improve the quality 
and sustainability of healthcare practice and provision have become a policy concern. In addition, the involvement of 
stakeholders in health policy decision-making has been advocated, as complex questions arise around the structure of 
funding arrangements in a context of limited resources. 


AIMS. Review the evidence regarding the outcomes of interventions used in ART for the treatment of infertility and 
comparative analysis of stakeholder involvement in the case of ART in Kazakhstan through the concepts of voice, choice 
and co-production, assessing the implications for stakeholder empowerment. 


METHODS. We are analysed meta-analyses and systematic reviews, the cornerstones of evidence-based medicine, 
pertaining to two important problems in in vitro fertilization: failed implantation and poor ovarian response to ovarian 
stimulation. Short-term outcomes included pregnancy, live birth, multiple gestation, and complications. Long-term 
outcomes included pregnancy and post-pregnancy complications for both mothers and infants. Using a case study of ART, 
deliberative engagements with a range of stakeholder groups were held on the topic of how best to structure the 
distribution of Kazakh public funding in this domain. 


RESULTS. Numerous interventions and procedures have been tried to facilitate implantation and to enhance the ovarian 
response to stimulation. Notwithstanding the fact that many clinical trials have been performed, very few procedures can 
as yet stand the critical test of evidence-based medicine. Our questions demonstrated stakeholders’ capacity to 
understand concepts of choice under resource scarcity and disinvestment, and to countenance options for ART funding 
not always aligned with their interests. Deliberations in each engagement identified concerns around ‘equity’ and ‘patient 
responsibility’, culminating in a broad preference for (potential) ART subsidy restrictions to be based upon individual 
factors rather than maternal age or number of treatment cycles. All groups advocated continued patient co-payments, 
with measures in place to provide treatment access to those unable to pay (namely, equity of access). 


LIMITS. Short terms period for analysis 


CONCLUSIONS. A plea is made for co-ordination between clinicians and reviewers and co-operation between infertility 
centres to combine their efforts to set up sufficiently powered clinical trials to arrive at more solid evidence for a number 
of interventions in in vitro fertilization programmes.Deliberative processes, engaging key stakeholders including patients, 
in decision making around potential disinvestment from low value health care, provides an avenue for contributing to 
policy making in a contentious area of health policy. It would be to the advantage of our patients and society in general if 
clinicians transcend their individual aspirations and co-operate to perform good clinical trials that can provide an 
evidence-based answer to the many problems that remain to be solved in ART. 
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86. Predictors of mortality in bloodstream infections caused by Enterobacteriaceae 
carbapenemase -producing: importance of combination therapy and the role of Antimicrobial 
Stewardship 


Confalonieri Massimo, Reboli Camilla, Confalonieri Corrado 


Piacenza Hospital 


BACKGROUND. The World Health Organization was requested by Member States to develop a global priority pathogens 
list of antibiotic-resistant bacteria to help in prioritizing the research and development (R&D) of new and effective 
antibiotic treatments. Future R&D strategies should focus on the discovery and development of new antibiotics 
specifically active against multidrug- and extensively drug-resistant Gram-negative bacteria. In the time of increasing 
resistance and paucity of new drug development there is a growing need for strategies to enhance rational use of 
antibiotics. Carbapenemase-producing Enterobacteriaceae (CPE) produce enzymes that can efficiently hydrolyse and 
confer resistance to most B-lactams, including the carbapenems. In addition, many CPE strains frequently carry additional 
genetic determinants that confer resistance to other non-8-lactam antibiotics, making these bacteria resistant to most 
antibiotic.ltaly is now witnessing an endemic situation of CPE. CRE, especially carbapenem-resistant K. pneumoniae, have 
a high potential to cause outbreaks in healthcare settings. Such outbreaks have been reported from several EU Member 
States, e.g. the Czech Republic, France, Germany, Greece, Italy, Spain and the UK. Risk factors for acquisition of CRE in 
healthcare settings are similar to those reported for acquisition of other multidrug-resistant bacteria. These include, for 
example, admission to an intensive care unit (ICU), long ICU stay, critical illness, invasive device use and prior 
antimicrobial therapy. International high-risk bacterial clones such as the KPC-producing K. pneumoniae ST258 have 
emerged. The spread of Klebsiella pneumoniae (Kp) strains that produce K.pneumonaie carbapenemases (KPCs) has 
become a significant problem, and treatment of infections caused by these pathogens is a major challenge for clinicians. 


AIMS. The outcome measured was death within 30 days of the first positive blood culture. Survivor and nonsurvivor 
subgroups were compared to identify predictors of mortality. Patient variables included age, sex, Charlson Comorbidity 
Index, underlying disease, immunosuppressive therapy, duration of index hospitalization, and BSI onset in an intensive 
care unit. We also considered histories of previous hospitalization, surgery, invasive procedures performed = 72 hours 
before BSI onset and antimicrobial therapy being administered = 30 days before BSI onset. Infection variables consisted 
of BSI presentation with septic shock, severity of illness at infection onset, and source infection. 


METHODS. In this monocentric retrospective cohort study, conducted in Piacenza Hospital, we examinated 60 patients 
with bloodstream infections (BSIs) caused by KPC-producing Kp isolates (KPC-Kp) diagnosed between 01 February 2014 
and 01 March 2016. The Vitek 2 automated system was used for isolate identification and antimicrobial susceptibility 
testing. Minimum inhibitory concentrations (MICs) were classified according to European Committee on Antimicrobial 
Susceptibility Testing (EUCAST). Statistic analysis: Test del chi-quadro Odds Ratio Test U - Mann Whitney Kaplan Meyer 
Curve 


RESULTS. A significant higher rate was observed among patients treated with monoterapy. In logistic regression analysis, 
30-days mortality was independently associated with septic shock at BSI onset; inadequate initial antimicrobial therapy; 
and high APACHE SCORE III. Postantibiogram therapy with a combination of tigecycline, colistin and meropenem was 
associated with lower mortality. 


LIMITS. Hospital is progressing computerization of Clinics Folders: this involves a difficulty in the collection dates for How 
in part attributable to the sheet only therapy and partly to the computerized database. Therefore the state will not be 
assessed All those Where folders extrapolation of dates goes back a Different Sources 


CONCLUSIONS. To improve survival, combined treatment with 2 or more drugs with in vitro activity against the isolate, 
especially those also including a carbapenem, may be more effective than active monotherapy. 
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87. Using social media to disseminate vaccine related information: an observational study. 


De Rinaldis Maria Francesca’, Lopalco Pietro Luigi’, Brioni Alessandro? 


1APSS Trento, *Universita degli Studi di Pisa, 7AULSS 9 Scaligera 


BACKGROUND. Facebook is increasingly used by Antivaccination activists to disseminate health information. There is 
robust evidence that a growing number of persons are seeking health related (including vaccinations) information on the 
internet. Therefore the world wide web, while is giving a common space for new antivaccination groups, is also a space 
for people seeking information and clarifications regarding the correct use of vaccines. Given the high likelihood of 
finding many unproven conspiracy theories on the world wide web, people are exposed to a real risk of disinformation.To 
analyze world wide web tendencies, to follow the dynamics and to understand the communicative mechanism of 
spontaneous web communities groupings via social media are important public health interventions. 


AIMS. To perform an analysis on relation dynamics between different groups sharing vaccine related information via 
Facebook. 


METHODS. Examination Facebook users profiles of “Comilva Association” and “VaccinarSi” followes. “Comilva 
Association” is the online community of the main Italian antivaccination group, while “VaccinarSi” website aims to 
disseminate correct information regarding vaccinations.The study was conducted on a sample of facebook profiles 
belonging to the two cited web pages. On the sampled profiled we collected information of sex, work, parental status, 
vaccination attitudes. An Analytical observational study was carried out, using a simple randomization of the total 
profiles extracted using Netvizz. 


RESULTS. The sample included 329 profiles from “Comilva Association” and 129 from “VaccinarSi” (5% of total users of 
each web site). Comilva association users were found to be 252 female, 72 male and 5 missing data. VaccinarSi users are 
94 female profiles, 24 male and 2 missing data. Age of users is known only for 21 profiles. Age span 20 to 49 for Comilva 
users with 9 out of 15 users included in their thirties. Data of parental status was missing for 27% of the sampling. In those 
known, results show 65% of parents in Comilva association followers against 35.6% in VaccinarSi Users. 43% of VaccinarSi 
profiles declared to be an health worker, against 5% in Comilva Association users.This information is unknown in 50% of 
VaccinarSi users and 38% of Comilva association users. Profiles included in the sample were analyzed with regards to 
vaccination orientations by gathering information on posts shared on facebook board and likes. A clear difference of 
orientations is showed by the performed analysis: Comilva association followers share a position against to vaccinations, 
while VaccinarSi users are pro vaccines. 


LIMITS. Due to privacy settings applied by users of Facebook apps (including Netvizz) it wasn’t always possible to obtain 
certain information. Age of Facebook profile holder was available only in a small percentage of our sample, being 
undisclosed for the majority of profiles due to users privacy settings. 


CONCLUSIONS. With this study we gained a deeper knowledge of profiles of persons against or uncertain about 
vaccinations. This may contribute to develop more appropriate communication strategies to deal with antivaccination 
positions and to promote evidence based reasons to accept vaccinations. 
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88. The informative value of research on quality of care for cancer patients. A scoping review 


Grilli Roberto, Chiesa Valentina*, Marchesi Cristina? 


Local health Authority of Reggio Emilia, Italy, 7?Department of Biomedical, Biotechnological and Translational Sciences, University of 
Parma, Italy 


BACKGROUND. Waste in clinical research occurs when clinicians’ and patients’ informative needs are not met, and 
research does not provide information useful to support their decisions. Similarly, resources are wasted in health services 
research (HSR) when issues relevant to the health policy making process are not addressed. Research on quality of care is 
the branch of HSR devoted to the identification of problems concerning health care delivery, and to the analysis of their 
main determinants, with the ultimate goal of providing relevant information to support the policy making process. 
Overuse of diagnostic and therapeutic procedures/interventions has been acknowledged as a policy issue of utmost 
importance. 


AIMS. To explore to what extent research on quality of cancer care is well equipped to meet policy makers’ informative 
needs, being a) oriented towards the identification of overuse, b) providing estimates of overuse prevalence, and c) 
providing information on its determinants (i.e. factors associated with patient unnecessary exposure to 
interventions/procedures 


METHODS. We adopted the scoping reviews’ methodology, an approach increasingly used when the goal Is to provide an 
overall description and analysis of the available literature in a field, area or on a specific topic, rather than to conduct a 
conclusive quantitative synthesis. Studies providing information on the rate of use of diagnostic or therapeutic 
interventions, procedures, services in cancer patients, published in english, from 2006 to 2016 and conducted in European 
countries were identified by searching electronic resource (Pubmed),references, and expert consultation. As we 
anticipated that the quantity of studies was likely to be large, to keep the task more manageable, we focused on studies 
concerning patients with breast, colorectal, lung, and prostate cancer, and restricted the time frame of our search as 
older papers were likely to describe patterns of care no longer fully representative of current clinical practice. 


RESULTS. Out of 1882 papers identified, 97 accounting for 89 studies met our eligibility criteria. Most of the studies (n=37) 
were on breast cancer, 26 on colorectal, and 8 and 9 on lung and prostate cancer, respectively. More than one cancer 
type was included in the remaning studies. Fifty studies (56%) described the patterns of use of the procedures and 
interventions considered through rates of compliance with local/regional/national guidelines, while in the remaining 
only descriptive rates were provided. Among those analysing compliance with guidelines, most of the time (n=32; 36%) 
rates assessed to what extent eligible patients actually received the recommended course of action, thus being more 
oriented towards the identification of under rather than over utilisation. Only 18 studies (20%) assessed whether 
procedures/interventions were used in the appropriate clinical indications, and 14 of them identified some degree of 
overuse. Overuse was much less frequently identified when only descriptive rates (6/39, 15%) or rates in eligible patients 
(6/32, 18%) were adopted. Overall, out of the 26 studies detecting overuse, only 16 provided estimates of its prevalence. 


LIMITS. Our search was limited to Medline, and focused only on studies conducted in European countries dealing with the 
most frequent cancer types. Nevertheless, we believe it provided a representative sample of the literature available this 
area 


CONCLUSIONS. Despite its policy relevance, economic impact and potential harm to patients, overuse is often 
overlooked by current HSR on quality of care for cancer patients in Europe, and most of the studies are more likely to be 


oriented towards the identification of underuse 
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89. Value-based healthcare: concepts and application 


Kostyuk Alexander?, Almadiyeva Alima’, Akanov Amangali*, Nurgozhin Talgat?, Aiypkhanova Ainur? 


+Republican Center for Healthcare Development, Astana, Kazakhstan,?Medical University, Astana, Kazakhstan,;Nazarbayev University, 
Astana, Kazakhstan 


BACKGROUND. World's best healthcare systems in the 21st century is characterized by evidence-based medicine (EBM), 
patient-centered care, and cost effectiveness. EBM involves clinical decisions being made by integrating patient 
preference with medical treatment evidence and physician experiences. Ministry of health Republic of Kazakhstan 
suggested value-based healthcare (VBHC) as the practice of health services based upon the patient-perceived value 
conferred by an intervention. VBHC starts with the best evidence-based data and converts it to patient value-based data, 
so that it allows clinicians to deliver higher quality patient care than EBM alone. The final goals of VBHC are improving 
quality of healthcare and using healthcare resources efficiently. The pressing of pharmaceutical manufacturers in the 
Kazakhstan is the emergent need to demonstrate the value of medicines for both the patient and the healthcare system 
as a whole. This study introduces the concepts and application of VBHC and suggests some strategies for promoting 
related research. 


AIMS. The purpose of this study is to how Kazakh healthcare provider and practitioners understand the concept of value- 
based healthcare (VBHC) and determine how can apply lessons from quality and outcomes standards commonplace 
within European healthcare systems to better demonstrate value to the evolving Kazakhstan healthcare system explore 


METHODS. We conducted a literature review using publicly available databases, and web-based searches to examine 
cases where manufacturers played a role in improving quality and outcomes in European healthcare systems. We then 
assessed the Kazakh healthcare landscape to identify opportunities for application of these findings, given the emerging 
focus on value following implementation Obligatory Medical Insurance. We conducted an open-ended interviews were 
used as the data-collection method and content analysis of the transcribed interviews was carried out. 


RESULTS. Participants’ understanding of VBHC focused on how value was created for the patient and on measuring 
medical outcomes and costs, although costs were to some extent put aside. To measure value for the patients, it was the 
health professionals’ perspective about what patient should value that dominated the understanding of the concept 
VBHC. VBHC was understood as a strategy to strengthen value innovations and to loosen the grip of economic control. 
Benchmarking was seen as a future possibility to develop value innovations.Changes in organizational culture were 
understood by participants as a need to change healthcare from being professional-centred to patient-centred. The way 
the concept was understood omits parts of the original concept. This has implications for whether or not the concept as it 
is described by the participants should be understood as VBHC according to the intentions of the strategy described. 
Demonstrating value and ensuring sustainability will require pharmaceutical manufacturers to work in partnership with 
payers, healthcare providers, and patients to ensure that the value of the treatments is fully realized and demonstrated to 
all stakeholders involved. 


LIMITS. No 


CONCLUSIONS. The European value-based experience serves as a positive model and offers numerous examples of both 
value demonstration and care models, including the development of successful payer-manufacturer partnerships. As the 
Kazakhstan progresses along this same value-focused path, there is a significant and necessary role that manufacturers 
can play to drive higher value care and demonstrate better outcomes. The development of outcome measures was 
predominantly based on the health professionals’ experiences, which is why the patients’ perspective needs to be 
strengthened. Further studies of the process of implementing VBHC are needed. 
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90. Effects of protein supplement combined with resistance exercise on body composition and 
physical function in elderly individuals with sarcopenic obesity_a systemic review and meta- 
analysis 


Liao Chun-De?, Tsauo Jau-Yih?, Chen Hung-Chou}, Liou Tsan-Hon* 


Taipei Medical University, Taipei, Taiwan, *National Taiwan University, Taipei, Taiwan 


BACKGROUND. Sarcopenic obesity has been characterized as simultaneously suffering low muscle mass and high body 
fat, which can be effectively improved by resistance exercise training (RET). Nutrient supply, such as protein supplement 
(PS), has been recommended for additional lean mass and strength regain in individuals receiving RET or in obese elderly 
individuals with energy restricted diet. Whether PS during RET exert any benefit on augmentation of muscular and 
functional performance in obese elder people remains unclear. 


AIMS. The purpose of this study was to identify the effect of PS in combination with resistance exercise on body 
composition and physical function for elder people with sarcopenic obesity. 


METHODS. A comprehensive search of online databases including PubMed, EMBASE, and Google Scholar databases etc. 
was performed to identify randomized controlled trials (RCT) that reported the efficacy of PS on lean mass gain, strength 
gain, and physical mobility in obese elder individuals with RET. 


RESULTS. Sixteen RCTs were included in the analysis. Nine trials had excellent methodological quality with a PEDro score 
of 9, whereas seven ones had a PEDro score of 8. Participants with additional PS had significantly greater gain in lean 
mass with a standard mean difference (SMD) of 0.58 (95% Cl 0.30, 0.86; p < 0.0001; 12 = 70%, p < 0.0001); and in leg 
strength with a SMD of 0.71 (95% Cl 0.37, 1.04; p < 0.0001; 12 = 59%, p = 0.009). 


LIMITS. Several limitations to our findings should be considered: first, based on varieties of protein supplement protocol 
(protein source, supplied amounts, timing of ingestion) and exercise regimes (training duration, training volume), it was 
difficult to endorse a definite conclusion for the effect of specific type of protein supplement on lean mass gain or 
strength gain; second, some of our included trials were small sample size, results of these study that reflected no 
significant treatment effect on lean mass may contribute negative effect size to the overall; third; limited studies 
regarding additional PS for obese elderly trained with resistance-type exercise were available. Some participants of our 
included studies were classified as overweight with a BMI cutoff point of 25 kg/m2; finally, most of our included studies 
had outcome followup up to six months immediately after intervention finished, no had a long-term followup after the 
end of intervention. 


CONCLUSIONS. Additional PS in combination with RET is effective for eliciting gains in lean body mass, appendicular or 
leg lean mass, and muscle strength, except a parallel gain in physical mobility or physical capacity, in obese elder adults. 
Clinicians should incorporate strategies for improving physical activity into their management of patients with sarcopenic 
obesity to maximize health status, especially for elder individuals who are overweight or obese. 
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91. How can we reduce vulnerability to misleading persuasion? 


Mansfield Peter 
Healthy Skepticism Inc 


BACKGROUND. One important barrier to evidence based health care is misleading persuasion e.g. by pharmaceutical 
companies. Many health professionals feel offended by the suggestion that they are vulnerable to being misled. Thus 
they respond with denial rather than appropriate behavior changes such as avoiding misleading persuasion and seeking 
more reliable information. 


AIMS. To understand why intelligent people are vulnerable to misleading persuasion and how to reduce their 
vulnerability. 


METHODS. Experience and a snowball literature search during 35 years as founder and director of the Medical Lobby for 
Appropriate Marketing Inc. then Healthy Skepticism Inc. 


RESULTS. Vulnerability to misleading persuasion is explained by the so-called dual process theories of (fast vs slow) 
human decision-making including the elaboration likelihood model of persuasion. Many persuasion techniques are 
triggers for heuristics (decision-making shortcuts). Heuristics enable us to make decisions quickly with little effort and 
little or no conscious awareness. Heuristics lead to good outcomes if the environment is safe and stable. However 
heuristics make us vulnerable to being misled by those who are able to trigger them. Many heuristics have been known 
for centuries as logical fallacies. Many people believe that their high intelligence and knowledge provide protection from 
being misled. The evidence shows otherwise. High intelligence can lead to overconfidence. Overconfident people are 
more vulnerable because they tend to rely on heuristics rather than taking more time to decide. The quality of human 
decision-making depends more on thoughtfulness: the tendency to take time to think carefully before reaching a 
conclusion. Thoughtful people will respond to a heuristic trigger by suspending judgment until they have considered 
additional information. Thus thoughtfulness overlaps with the concept of healthy scepticism which may be the best 
defense against misleading persuasion. Tests for measuring thoughtfulness are being developed. Unlike intelligence, 
thoughtfulness can be learned. There are 3 methods for decreasing vulnerability to misleading persuasion that are worth 
further testing. The first is education to increase thoughtfulness/healthy scepticism. The second is to mislead people then 
show them that they have been misled to dispel the illusion of invulnerability. The third is to draw an analogy between 
the paradigm shift to acceptance that doctors can unknowingly carry bacterial infections to acceptance that doctors can 
unknowingly be infected with bias. This analogy suggests that angry denunciations a la Semmelweis may be ineffective 
but education about the psychology of persuasion analogous to education about the germ theory of disease a la Lister 
may work better. 


LIMITS. My snowball literature search was limited to work published in English and may have missed important work. 
There are many competing so-called dual process theories subject to ongoing debate. The 3 proposed methods for 
decreasing vulnerability have not yet been adequately tested. They may work better in combination. 


CONCLUSIONS. The next step is to develop and test educational interventions for decreasing vulnerability to misleading 
persuasion by increasing careful decision-making. 1. Spurling GK, Mansfield PR, Montgomery BD, et al. Information from 
Pharmaceutical Companies and the Quality, Quantity, and Cost of Physicians’ Prescribing: A Systematic Review. PLoS Med 
2010 Oct 19;7(10) 2. Mansfield P. Accepting what we can learn from advertising's mirror of desire. BMJ 2004 Dec 
18;329(7480):1487-8 3. Chaiken S, Trope Y. Dual-process theories in social psychology. Guilford Press 1999 4. Kahneman D. 
Thinking, Fast and Slow. Macmillan 2011 5. Evans J, Stanovich KE. Dual-process theories of higher cognition: Advancing 
the debate. Perspect Psychol Sci 2013 (8)223-241, 263-271. 6. Hamblin CL. Fallacies. Vale Press 1970 7. Clever people "are 
easier to con" Ultrascan 2008 www.ultrascan-agi.com/public_htmV/html/419_Victims_and_Global_Losses.html 8. 
Konnikova M. The Confidence Game. Why We Fall for It... Every Time. Viking 2016 9. Stanovich KE. Rationality and the 
Reflective Mind. Oxford University Press 2011 10. Mansfield PR. Garai’s challenge: The 40th anniversary of the inspiration 
for Healthy Skepticism. HS Int News 2003(10) www.healthyskepticism.org/global/news/int/hsin2003-10 11. Hurtt RK. 
Skeptical about skepticism: Instrument development and experimental validation. Dissertation. David Eccles School of 
Business, University of Utah 1999 12. Stanovich KE. The Rationality Quotient: Towards a Test of Rational Thinking. MIT 
Press 2016 13. Mansfield PR, Lexchin J, Wen LS, et al. Educating Health Professionals about Drug and Device Promotion: 
Advocates’ Recommendations. PLoS Med 2006;3(11): e451 14. Gérvas J, Mansfield PR. Physicians, the industry and 
population health. J. Epidemiol. Community Heal. 2009;63(10):773-774 
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92. Assessment of the prevalence of overuse in breast cancer patients’ follow up 


Marino Massimiliano, Mangone Lucia, Marchesi Cristina, Grilli Roberto 


Local health Authority of Reggio Emilia, Italy 


BACKGROUND. Research on quality of care and its determinants has been shown to be frequently overlooking the 
problem of health services overuse, despite its increasing policy relevance. Lack of reliable and detailed information on 
the processes of care actually adopted in caring for individual patients has been highlighted as one of the reasons for 
such a failure of research in meeting policy makers’ informative needs. In this study, we explored to what extent reliance 
on data routinely available at the local level could allow an assessment of the prevalance of overuse in the area of breast 
cancer care. Indeed, cancer care cannot claim to be untinged from the risk of exposing patients to “too much care”, that is 
to diagnostic procedures or therapeutic interventions that do not meet their actual health neeeds. The follow-up of 
breast cancer patients at low risk of recurrence is a possible case in point: despite guidelines recommendations, clinical 
practice has been shown to be more aggressive than needed, in terms of type and frequency of use of the diagnostic 
procedures adopted 


AIMS. In this study, relying on data from the local cancer registry, linked with adminsitrative databases, we assessed the 
prevalence of overtulisation of procedures occuring during the follow-up of breast cancer patients after their primary 
surgical and medical treatment 


METHODS. A cohort of 970 women with stage | (Tis, TO<=2 cm, NO, MO) breast cancer diagnosed over the period 2010- 
2013 was identified in the Reggio Emilia Cancer Registry. Information on type and number of diagnostic procedures 
undergone by those women during follow up after their primary treatment was then gathered from administrative 
databases. In particular, we focused on clinical examinations (performed by different specialists), radiography, 
echographies, PET, bone scintigraphy and densitometry, CEA/CA 15.3. The observed pattern of use of these procedures 
was compared with what currently recommended by practice guidelines, in order to identify the degree of overuse (if 
any). Costs entailed by following up individual patients were calculated relying on the tariffs officially adopted at the 
Emilia-Romagna regional level. As for the statistical analyses, beyond usual descriptive statistics, univariate analyses 
were conducted to identify patient characteristics associated with overuse. Then, a quantile regression model was 
implemented with the total cost as dependent variable and year of diagnosis, age and geographic area of patient's 
residency as predictive variables. 


RESULTS. The median number of procedures performed in the cohort of 970 breast cancer patients was 23 (mean 22+12). 
Overall, only 3% (n=29) women did not undergo any clinical examination or tests over the first 3 years after diagnosis, 
while 88% (n=857) and 12% (n=113) receveid more and less tests than what recommended, respectively. Overall, non 
compliance with guidelines on breast cancer follow up in low risk patients was observed both in frequency and type of 
procedures adopted. No substantial difference was observed between women in different age groups. Women living in 
relatively more remote (i.e. mountains/hills) geographic areas of the province were exposed to a lower number of clinical 
examinations and procedures (median 17 vs 24, respectively). When factors associated with overuse were explored 
through quantile regression, geographic area and year of diagnosis emerged as associated to the numer of tests 
undergone. Indeed, over the three -year study period a trend towards a decreasing cost of overuse was observed (from a 
median cost €645 in 2010 to €569 in 2013). Overall, the patterns of use of the diagnostic procedures observed accounted 
for €734,000 (median cost per patient: €717,00; range 0-€4492), 79% of which explained by some degree of overuse. 


LIMITS. Although, we selected a population of breast cancer patients considered to be at a very low risk of recurrences, 
we could not discriminate whether some of the procedures performed were actually justifiable on a clinical ground (i.e. 
performed when a recurrence was legitimately suspected). 


CONCLUSIONS. Reliance both an cancer registry and administrative data allowed and estimate of the prevalence of 
overuse in the clinical management of breast cancer patients follow-up. Extrapolating our figures to the regional Level, 
assuming, according to available epidemiological figures, that approximately 2200 low risk stage | breast cancer patients 
are diagnosed every year, would imply that overall €3 milions euro are yearly wasted through an excessively intensive 
follow-up. 
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93. Troubled Evidence? Tracking Excess Significance, Cherry-Picking, and Premature Closure 


Martin Janet, Medici Team On Behalf Of The 


MEDICI Centre, Schulich School of Medicine & Dentistry, at Western University, London, CANADA 


BACKGROUND. Effective evidence-based decision making relies heavily on the quality of the available evidence base. 
Better understanding of the validity and relevance of the evidence base, as well as trends over time, will be imperative in 
order to inform current deficiencies and future research agendas. 


AIMS. To quantify the validity and relevance of published evidence, and to explore trends over time. 


METHODS. A meta-epidemiologic survey of the literature was performed, covering over 50 years of published 
randomized clinical trials from 15 medical journals. 


RESULTS. While the number of randomized trials has increased significantly over the years, the majority failed to report 
on clinically-relevant outcomes. Furthermore, the mean size of randomized trials has remained small ( 


LIMITS. These results should be interpreted in light of the strengths and limitations. Strengths include the breadth of years 
and medical specialties covered in this meta-epidemiologic survey. Weaknesses include the focus on only randomized 
trials, and therefore conclusions may not generalize to other study designs. 


CONCLUSIONS. Despite persistently small sample sizes, and despite few clinically-relevant outcomes being measured, 
the majority of randomized trials report positive conclusions and advocate for the superiority of the proposed 
intervention. Future research is urgently required to increase awareness of these detractions to the validity and relevance 
of the evidence base, and to develop transparent safeguards against these biases during design, conduct, and reporting of 
clinical trials. 
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94. Improving the implementation of stroke thrombolysis: a cluster randomized controlled trial 


Paul Christine’, Attia John’, Deste Catherine’, Ryan Annika’, Kerr Erin’?, Henskens Frans}, Levi Chris? 


University of Newcastle, 7#Hunter New England Area Health Service 


BACKGROUND. One of the three effective interventions in the acute phase of stroke care is appropriate thrombolytic 
therapy with tissue plasminogen activator (tPA). This treatment has low implementation rates internationally. 
Collaborative quality improvement interventions are often used to change practice but rarely subjected to randomised 
trials. 


AIMS. Our objective was to test the effectiveness of a multi-component multidisciplinary collaborative approach 
compared to usual care as a Strategy for increasing thrombolysis rates for stroke patients 


METHODS. A cluster randomised controlled trial was conducted in 20 hospitals across three Australian states. Hospitals 
were stratified based on baseline thrombolysis rate and randomly allocated within strata. The intervention was based on 
behavioural theory and analysis of the steps, roles and barriers relating to rapid assessment for thrombolysis eligibility, 
and involved a comprehensive range of strategies addressing individual-level and system-level change at each site. The 
primary outcome was the difference in tPA rates between the two groups post-intervention. The secondary outcome was 
the proportion of tPA treated patients in both groups with good functional outcomes (modified Rankin Score (mRS2), 
compared to international benchmarks. 


RESULTS. The collaborative quality improvement intervention was implemented at intervention hospitals over a 16 
month period with final follow-up concluding in December 2015. Average attitude score at the intervention sites 
increased by an additional 1.5 units (0.4 to 2.6) compared to control sites (p 


LIMITS. The use of cluster-level data limits the ability of the study to explore patient-related effects on the study. 


CONCLUSIONS. The multidisciplinary collaborative quality improvement approach only had a small effect on attitudes 
and a transitory effect on stroke care practice, despite the major investment required to enact such intervention 
approaches 
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95. Outcomes of a diabetes care implementation cluster trial in Indigenous health 


Paul Christine’, Ishiguchi Paul?, Shaw Jonathan’, Deste Catherine?, Eades Sandra? 


University of Newcastle, *Baker IDI Heart and Diabetes Institute, 7Australian National University 


BACKGROUND. Type 2 diabetes is one of the key health problems in the Australian Indigenous population. Aboriginal 
Community Controlled Health Services (ACCHSs) are primary care service delivery settings where there is opportunity to 
develop partnerships to reduce the current evidence-practice gap in type 2 diabetes testing, monitoring and control. 


AIMS. This study (the ADACC trial) aimed to examine the effectiveness of a tailored collaborative model in achieving 
adherence to best practice guidelines for type 2 diabetes care 


METHODS. A cluster-randomised wait-control design was used in 18 ACCHSs (9 intervention and 9 wait control). Services 
were stratified on rurality and size and randomly allocated within strata. Electronic data for diabetes testing and care for 
adult Indigenous patients at the 18 services was collected from July 2010 to October 2015. The intervention (May 2014- 
April 2015) included strategies to 1) Achieve organisation and provider commitment to reduce the evidence practice gap; 
2) Explore the evidence-practice gap and potential solutions; 3) Collaborate to achieve change through local and 
combined intervention service workshops and other methods of communication; and 4) Ongoing communication, 
feedback of local performance data and collaborative problem solving. The study outcome measures were guideline- 
appropriate diagnostic testing for type 2 diabetes; guideline-appropriate monitoring for glycaemic control and blood 
lipids; guideline-appropriate glycaemic control and blood-lipid control. Staff surveys were also completed. 


RESULTS. Mixed effects logistic regression (at post-test adjusted for stratification, baseline values and correlation within 
ACCHSs) found a non-significant effect for each of the primary outcomes of testing, monitoring and control. There were 
significant differences between intervention and control group services at baseline in relation to each outcome variable. 
Some patient groups were more likely to receive each type of care. For example, females, older patients and those with a 
high number of service visits were more likely to be tested for diabetes (p 


LIMITS. The use of cluster-level data limits the ability of the study to explore patient-related effects on the study 


CONCLUSIONS. The tailored collaborative approach was not effective in changing practice with regard to diabetes 
testing and management in ACCHSs 
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96. Embedding smoking cessation support in cancer care clinics in Australia 


Paul Christine’, Sherwood Emma’, Freund Megan’, Dadich Ann’, Meiser Bettina?, Taylor Natalie*, Shaw Tim®, 
Young Jane®, Day Fiona®, Aranda Sanchia’ 


1University of Newcastle, Western Sydney University, 7University of NSW, ‘Macquarie University, *University of Sydney, ‘Calvary Mater 
Newcastle Hospital, 7Cancer Council Australia 


BACKGROUND. Despite the evidence for improved outcomes of quitting smoking at or after a cancer diagnosis, the 
implementation of smoking cessation care in this settings is limited 


AIMS. This pilot project aimed to i) identifying the capacity of NSW hospitals to implement system-wide smoking 
cessation advice and support; ii) identifying the proportion of oncology staff who routinely delivered cessation advice and 
support; iii) exploring cancer patient perceptions and experiences of receiving cessation advice and support; iv) pilot- 
testing the feasibility of a cessation model at one site and v) obtaining preliminary consensus on the key components of 
cessation care for cancer patients. 


METHODS. Electronic surveys of 34 cancer care administrators or coordinators; 193 oncology staff (medical, nursing and 
allied health) and 281 cancer patients across seven cancer clinics. A pilot implementation plan and patient pathway were 
developed based on the survey data. Qualitative interviews with staff and patients occurred after implementation. A 
modified Delphi process was used to develop care principles and key cessation messages 


RESULTS. Stakeholder data indicated a lack of cancer-patient-specific policies on smoking cessation (0%), a lack of 
electronic support and confusion regarding current cessation care provision. Approximately three-quarters of 
stakeholders reported that existing services did not meet the needs of cancer patients who wish to quit smoking. 
Oncology staff indicated asking just under half of their patients about their smoking status for (48% inpatients, 43% 
outpatients), and few reported referring their patients to external supports to help them quit. Both oncology staff and 
patients indicated high (over 90%) levels of support for cessation care for cancer patients, and patients reported 
relatively low levels of practical cessation support. The implementation pilot identified a series of ongoing challenges. 
The outcomes of the Delphi process included a preference for delaying the timing of the discussion of cessation, in 
contrast to the limited evidence from oncology cessation trials. 


LIMITS. Full-scale randomised trials are required to identify whether our pilot approach will be effective in practice in 
changing smoking cessation rates among cancer patients. 


CONCLUSIONS. Both patients and providers in the oncology outpatient setting endorse the need for and value of 
providing smoking cessation support for patients who continue to smoke. There is a need for rigorous trials of evidence- 
implementation strategies to embed smoking cessation care in oncology settings 
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97. Including People with Dementia in Research: Evidence to support the appropriate regulation of 
research 


Ries Nola 


University of Technology Sydney 


BACKGROUND. Dementia is one of the most pressing health issues worldwide, especially in countries with rapidly aging 
populations. Research is crucial to advancing knowledge about dementia, including its causes, possible therapeutic 
interventions, and ways to improve the quality of life of people living with dementia. Yet studies often exclude people 
with cognitive impairment. Researchers who propose to include people with dementia in research report obstacles 
arising from the complexities of obtaining ethical and legal approval, including issues of capacity, and the challenges of 
seeking consent from substitute decision-makers. As a result, dementia research has not kept pace with the burden of the 
disease. 


AIMS. Using Australia as a case study: (1) to examine current legal and ethical rules governing the inclusion in research of 
people with cognitive impairment and identify areas for reform to remove unjustifiable barriers to research participation; 
(2) to determine in a sample of dementia researchers in Australia, experiences with including people with cognitive 
impairment, including practices in relation to seeking consent, the use of advance research directives, and experiences 
with human research ethics committees; and (3) to determine in a sample of people aged over 60 attending outpatient 
clinics, their willingness to participate in research activities if they were to develop dementia and their views on 
substitute decision makers and advance research directives. 


METHODS. (1) Documentary analysis of laws and research ethics guidelines; (2) A cross-sectional, national survey of 
dementia researchers in Australia with follow-up qualitative interviews; and (3) A survey of older people attending 
outpatient clinics in a large metropolitan hospital. 


RESULTS. In Australia, the participation in research of people with cognitive impairment is governed by a national ethics 
statement and a patchwork of state and territorial laws that establish rules for how decisions concerning health and 
personal matters should happen for a person who lacks capacity. The national ethics document states that people with 
cognitive impairment are entitled to participate in research and encourages the use of advance research directives. In 
practice, however, researchers report significant ethical and legal barriers to recruiting people with dementia into 
studies. Relevant laws vary in important ways across the country. Laws in some jurisdictions set out detailed statutory 
rules governing approvals and substitute decision-making for research involving people who lack capacity to give their 
own consent. In some cases, statutes impose greater restrictions than what is laid out in the national ethics statement and 
require a legal tribunal to approve a study that has already been approved by a human research ethics committee. A 
further complexity is that statutes vary in how they define research, meaning some types of research (such as clinical 
trials) are covered by legislation and other types (such as observational studies) are excluded from statutory rules. 
Statutes generally allow people, when they have capacity, to designate individuals who will act as decision makers for 
health and personal matters should the appointer lose capacity in the future. Yet some statutes preclude a designated 
decision maker from making choices about certain types of research. Many older people express a willingness to be 
involved in research should they develop dementia in the future and are interested in the opportunity to record their 
wishes on health-related matters in advance directives. Yet advance research directives are rarely used in practice. 


LIMITS. The research focuses on Australia; comparative study of the legal and ethical governance of research in other 
countries is needed. Some data collection is still underway, but will be completed by mid 2017, allowing full results to be 
presented by October. 


CONCLUSIONS. In Australia, the gap between ethical guidelines and laws has no rational foundation, precludes a 
consistent approach to research governance and participation, and hinders research that seeks to include people with 
impaired capacity, especially multi-jurisdictional studies. Researchers and human research ethics committees require 
further training on strategies to support the appropriate inclusion of people with dementia in research. This work can be 
bolstered by the evidence of community support for more opportunities for people with dementia to participate in 
research. 
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98. Evidence-Based Healthcare as Inspiration for Evidence-Based Law 


Ries Nola?, Shelley Jacob? 


University of Newcastle, Western University 


BACKGROUND. The evidence-based movement has had a transformative influence on research and practice in 
healthcare. Laws play a central role in structuring healthcare systems, regulating health professions and giving legal 
effect to desired health policy objectives. Yet, as a field of research and practice, law-making is still dominantly 
‘eminence based’, relying on the opinions of authority figures, not ‘evidence-based’, relying on robust and thorough 
research evidence. 


AIMS. To conceptualise an evidence-based initiative that brings empirical and experimental research methods to 
understanding and advancing law as a vital tool to support health system and practice reforms. 


METHODS. A scoping review of literature on four themes: (1) the use of scientific evidence in law-making; (2) empirical 
research into practices of legal institutions; (3) experimental innovations in law-making; and (4) values and politics in 
evidence-based law-making. The literature review and analysis focuses on law-making on health-related issues. Themes 
were explored across a range of academic and grey literature from different disciplines, notably health, law and public 
policy. 


RESULTS. (1) It is recognised that scientific evidence plays a critical role in litigation, legislation and policy making. 
Scientific literacy is often low among legal institutional actors (such as judges and lawyers) and there are gaps in 
understanding how to improve evidence-informed law-making. This is particularly evident in areas relying on novel uses 
of science, such as using functional magnetic resonance imaging (fMRI) to determine consciousness in patients in a 
persistent vegetative state. (2) The theory and practice of empirical legal research remains underdeveloped. There is 
paucity of research on the impact of law in the ‘real world’ and how it acts as a barrier and enabler to the translation of 
health evidence into policy and practice. (3) An emerging area of research considers the use and adaptation of scientific 
methods to law. Examples include field experimentation in the study of laws and policies, randomised controlled trials 
involving legal institutional actors, and experimental legislation. However, the lack of a tradition of experimentation in 
law and legal institutions must be overcome to enable intervention studies of law as a tool to achieve health goals. (4) 
Evidence-based healthcare has been criticised for insufficient attention to the pervasive influence of values and politics 
on the production, dissemination and use of evidence. An evidence-based movement of law for health must confront 
politics directly. This is especially important given the increased demand for and acceptance of different modalities of 
healthcare delivery, incluing those associated with complementary and alternative medicine (e.g., homeopathy, 
naturopathy), integrative medicine, and Indigenous healing. 


LIMITS. Only literature published in English was included in the review. 


CONCLUSIONS. For laws to be used effectively as a tool to translate health evidence into practice, greater attention is 
needed to the processes involved in law-making. Legal institutional actors need greater scientific literacy and scientific 
methods can be adapted to legal experimentation, such as trials or pilots of new laws to support health programs and 
policies. The experiences and lessons of the evidence-based movement in healthcare can inspire an evidence-based 
movement for law and practice. 


Corresponding author: Ries Nola — University of Technology Sydney — Email: nola.ries@newcastle.edu.au 


tt] GIMBE 110 


EVIDENCE FOR HEALTH 


The ecosystem of evidence 
wy Connecting generation, synthesis and translation 


99. The effectiveness of post-acute care for the patients with cerebrovascular diseases: a 
systematic review 


Sun Mei-Hua?, Hsu Teh-Fu2, Yu Shut 


1National Yang-Ming University, Taipei Veterans General Hospital 


BACKGROUND. According to a recent WHO review of the literature, populations around the world are rapidly ageing. 
Between 2015 and 2050, the proportion of the world's population over 60 years will nearly double from 12% to 22%. By 
2050, the world’s population aged 60 years and older is expected to total 2 billion (World Health Organization, 2017). As 
aging is associated with increased utilization of healthcare services, the demographic trends present a growing problem. 
These issues presents major challenges for has been growing economic pressure on healthcare systems to contain and/or 
reduce costs, which has resulted in shorter duration of hospital admissions(Mabire, Dwyer, Garnier, & Pellet, 2016). In both 
developing and developed countries, chronic diseases are significant and costly causes of disability and reduced quality 
of life. Nearly 800,000 (approximately 795,000) people in the United States have a stroke every year, with about three in 
four being first-time strokes. Stroke is the No. 5 cause of death in the United States, killing nearly 130,000 people a year 
(128,978). Stroke is a leading cause of long-term disability and the leading preventable cause of disability (American 
Heart Association, 2016). Healthcare services provided after hospital discharge are referred to as post-acute care services 
and are designed to support patients in their transition from the hospital to home and in their pursuit of achieving the 
highest level of functioning possible (Winstein et al., 2016). 


AIMS. The objective of this systematic review is to identify, appraise and synthesize the best available evidence on the 
effectiveness of post-acute care for the patients with cerebrovascular diseases and to assess the relative impact of 
individual components of post-acute care strategies. 


METHODS. A three-step search strategy was used in this review. An initial limited search in CINAHL and MEDLINE 
databases was first performed, followed by an analysis of the text words contained in the title and abstract, and of index 
terms used to describe the article. A second search using all identified keywords and index terms was then undertaken 
across all included databases. Third, the reference list of all identified reports and articles was assessed to identify further 
additional studies. The databases searched included Cochrane Database of Systematic Reviews (CDSR), Database of 
Abstracts of Reviews of Effects (DARE), JBI EBP Database, CINAHL, MEDLINE, PubMed, CEPS. The search for unpublished 
studies included Google Scholar, Pro-Quest, Network Digital Library of Thesis and Dissertations, Trip Database and the 
websites of relevant associations such as cerebrovascular diseases Online. Initial keywords to be used will be: 
cerebrovascular diseases, cerebrovascular disorders, stroke, cerebral infarction, brain infarction, post-acute care, 
subacute care, transitional care, intermediate care, effectiveness. 


RESULTS. Ongoing. Preliminary results are presented that communication and coordination among team members are 
paramount in maximizing the effectiveness and efficiency of rehabilitation and underlie this entire guideline. 


LIMITS. In Taiwan, Ministry of Health and Welfare announced and promoted of “ The post-acute care-cerebrovascular 
diseases (PAC-CVD) pilot program" from 2014 to current. Consider the cultural differences, whether the relevant strategy 
applicable to Taiwan, remains to be discussed. 


CONCLUSIONS. This is an important issue to estimate the need for post-acute care (PAC) and to determine the factors 
related to the best available evidence on the effectiveness of post-acute care for stroke patients in Taiwan. We expected 


to provide the experience sharing and as a reference of the PAC-CVD. 


Corresponding author: Sun Mei-hua — Far Eastern Memorial Hospital — Email: sunmh103@gmail.com 


tt] GIMBE 111 


EVIDENCE FOR HEALTH 


The ecosystem of evidence 
wy Connecting generation, synthesis and translation 


100. Structural Equation Modeling in Medical and Health Sciences Research: Strengths and 
Limitations 


Violato Claudio 


Wake Forest School of Medicine 


BACKGROUND. Structural equation modeling (SEM) is a set of statistical techniques used to measure and analyze the 
relationships of observed and latent variables. Similar but more powerful than regression analyses, it examines linear 
causal relationships among variables, while simultaneously accounting for measurement error. Its applications range 
from analysis of simple relationships between variables to complex analyses of measurement equivalence for first and 
higher-order constructs. It provides a flexible framework for developing and analyzing complex relationships among 
multiple variables that allow researchers to test the validity of theory using empirical models. Perhaps its greatest 
advantage is the ability to manage measurement error, which is one of the greatest limitations of most studies. Although 
its application has been seen in many disciplines, it has yet to be extensively used in medical research and epidemiology. 


AIMS. The purpose of the present paper is to consider the potential advances that SEM can make in medical and health 
sciences research and provide a five step approach to implementing SEM research in epidemiology and medical research. 


METHODS. The basic principles of SEM as well as the procedures for creating, identifying, estimating, and evaluating 
structural models are outlined. 


RESULTS. Though it has extensive capabilities as a statistical research tool, SEM has been under-utilized in medical and 
health sciences research. To facilitate its use we explicate a five step procedure for applying SEM to research problems. 
Step 1: Identify the Research Problem: The researcher develops hypotheses about the relationships among variables that 
are based on theory, previous empirical findings or both. Step 2: Identify the Model: For each parameter in the model to 
be estimated, there must be at least as many observations (i.e., variance and covariance values) as model parameters 
(e.g.,, path coefficients, measurement error) . Step 3: Estimate the Model: The three primary estimation procedures are 
least squares (LS), maximum likelihood (ML) maximizes the amount of variance in the model and is more robust to non- 
normal data distributions and asymptotically distribution free (ADF) estimation procedures. Step 4: Determine the 
Model’s Goodness of Fit: These estimation procedures determine how well the model fits the data. Fitting the latent 
variable path model involves minimizing the difference between the sample covariances and the covariances predicted 
by the model. Step 5: Re-specify the Model if Necessary: To obtain improved fit results, the above sequence of steps are 
repeated until the most succinct model explains the most variance (i.e., principle of parsimony). Three examples of how 
SEM has been utilized in medical and health sciences research are presented. 


LIMITS. SEM requires a very well developed theory and empirical evidence, and, therefore, is limited in its applicability. 
When SEM is used for exploratory purposes, model fit may be more a function of statistical fit than theoretical fit. 
Furthermore, if the model is misspecified due to weak theory, unclear hypotheses or poor study design, the causal 
relationships between the variables will be misinterpreted. SEM requires considerable theoretical and statistical 
sophistication by researchers. 


CONCLUSIONS. SEM is a theory strong approach. Researchers employing this technique model hypothesized (latent) 
variables and more importantly, model the indirect and direct relationships between the latent and observed variables. It 
supersedes other multivariate analyses because it can model the relationships between “error free” latent variables by 
partialing out measurement error from multiple, imperfectly reliable indicators. Therefore, SEM can be used for a number 
of research designs. When many considerations are given to research planning, SEM can provide a new perspective on 
analyzing data and potential for advancing research in medical and health sciences. 
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101. A Pilot Study on Introduction of Evidence based Teaching Resources into Bilingual Teaching 
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Beijing University of Chinese Medicine 


BACKGROUND. Nowdays, there are a mount of evidence-based resources emerging into education field, especially in 
west nursing field. on the other hand, in China, bilingual nursing teaching is emphasied with a high degree all the time. 
And one of the main problem is lack of suitable, scientific, original textbooks. 


AIMS. To explore the whole process of introduction of evidence based teaching resource — Facilitating client centered 
learning, thereby to provide thoughts to peers who commit themselves to evidence-based education field. 


METHODS. Based on curriculum team’s four times of discussion, we designed to involve evidence based teaching 
resource (original English version) into our bilingual course. The reform was involved not only in teaching content, but 
teaching form and finally arranged in examination. And we applied mixed (qualitative and quantitative) reseach method 
to evaluate the results. 


RESULTS. After one session of teaching practice, the average score of examination only related to this part was 90+5 and 
evaluation average score from students to teacher was 97+3. On the other hand, both students and teacher gained 
confidence from the teaching content. 


LIMITS. We didn't design a controll class (another class with traditional teaching content) to compare the results. The 
students in this research is with particularity that has basic evidence based knowledge and techniques. Therefore, if 
nursing educationer who intend to apply evidence based teaching resources to their curriculum should be cautiously. 


CONCLUSIONS. Through this study, on one hand, our textbook has been supplemented, on the other hand, both teacher 
and students’ confidence has been increased due to scientific, original and updating teaching resources. Hence, more 
deep researches we are going to conduct in the near future. 
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